Multi-threat robustness remains a fundamental challenge in deep learning. Although joint adversarial training (JAT) is widely adopted, it suffers from negative transfer under heterogeneous threats, particularly between $\ell_p$-bounded and semantic attacks. Through first-order gradient analysis, we formalize this as gradient incompatibility and theoretically establish the necessity of decoupled optimization. We further reveal that these conflicting threats exhibit separable spectral characteristics in the frequency domain. Motivated by this observation, we propose Threat-aware Frequency Decoupling (TaFD), a two-stage defense framework that reformulates JAT as a frequency-domain divide-and-conquer paradigm. TaFD first discovers latent threat domains via unsupervised clustering of attack spectral prototypes and trains a lightweight classifier for inference-time threat domain identification. Conditioned on the prediction, TaFD employs a Frequency-Conditional Convolution that learns threat-domain-specific spectral masks and routes each sample to the corresponding expert, enforcing structural parameter separation and alleviating optimization conflicts. We validate TaFD on three representative image-classification benchmarks (CIFAR-10, CIFAR-100, and Tiny-ImageNet) and on two representative architectures (the convolutional ResNet and the hybrid-transformer MobileViT). Extensive results demonstrate that TaFD achieves more balanced robustness against heterogeneous attacks than existing JAT and frequency-domain baselines, improving average robust accuracy by approximately 11\% over the strongest baseline while maintaining leading clean accuracy.

翻译：多威胁鲁棒性仍然是深度学习中的基础挑战。尽管联合对抗训练（JAT）被广泛采用，但在异构威胁下（尤其是$\ell_p$有界攻击与语义攻击之间），该方法存在负迁移问题。通过一阶梯度分析，我们将此形式化为梯度不兼容性，并从理论上证明了解耦优化的必要性。进一步研究发现，这些冲突的威胁在频域中表现出可分离的频谱特性。基于这一观察，我们提出威胁感知频率解耦（TaFD）——一种将JAT重构为频域分治范式的两阶段防御框架。TaFD首先通过攻击频谱原型的无监督聚类发现潜在威胁域，并训练轻量级分类器用于推理时威胁域识别。根据预测结果，TaFD采用频率条件卷积学习威胁域特异性频谱掩码，将每个样本路由至对应专家模型，从而实现结构参数分离并缓解优化冲突。我们在三个代表性图像分类基准（CIFAR-10、CIFAR-100和Tiny-ImageNet）及两种典型架构（卷积ResNet与混合Transformer MobileViT）上验证了TaFD。大量实验表明，相较于现有JAT和频域基线方法，TaFD在异构攻击下实现了更均衡的鲁棒性，在保持领先干净准确率的同时，平均鲁棒准确率较最强基线提升约11%。