Adversarial robustness, which primarily comprises sensitivity-based robustness and spatial robustness, plays an integral part in achieving robust generalization. In this paper, we endeavor to design strategies to achieve universal adversarial robustness. To achieve this, we first investigate the relatively less-explored realm of spatial robustness. Then, we integrate the existing spatial robustness methods by incorporating both local and global spatial vulnerability into a unified spatial attack and adversarial training approach. Furthermore, we present a comprehensive relationship between natural accuracy, sensitivity-based robustness, and spatial robustness, supported by strong evidence from the perspective of robust representation. Crucially, to reconcile the interplay between the mutual impacts of various robustness components into one unified framework, we incorporate the \textit{Pareto criterion} into the adversarial robustness analysis, yielding a novel strategy called Pareto Adversarial Training for achieving universal robustness. The resulting Pareto front, which delineates the set of optimal solutions, provides an optimal balance between natural accuracy and various adversarial robustness. This sheds light on solutions for achieving universal robustness in the future. To the best of our knowledge, we are the first to consider universal adversarial robustness via multi-objective optimization.

翻译：对抗鲁棒性（主要包括基于敏感性的鲁棒性和空间鲁棒性）在实现鲁棒泛化中起着不可或缺的作用。本文致力于设计实现通用对抗鲁棒性的策略。为此，我们首先研究了相对较少探索的空间鲁棒性领域。随后，通过将局部和全局空间脆弱性统一到单一的空间攻击与对抗训练方法中，我们整合了现有的空间鲁棒性方法。此外，我们基于鲁棒表征的视角提供了有力证据，全面揭示了自然精度、基于敏感性的鲁棒性与空间鲁棒性之间的关系。关键在于，为将不同鲁棒性组分间的相互影响协调到统一框架中，我们在对抗鲁棒性分析中引入了\textit{帕累托准则}，从而提出了一种新颖策略——帕累托对抗训练，以实现通用鲁棒性。由此产生的帕累托前沿描述了最优解集，在自然精度与多种对抗鲁棒性之间实现了最优平衡。这为未来实现通用鲁棒性提供了思路。据我们所知，这是首次通过多目标优化来考虑通用对抗鲁棒性的研究。