Deep neural networks (DNNs) are sensitive to adversarial examples, resulting in fragile and unreliable performance in the real world. Although adversarial training (AT) is currently one of the most effective methodologies to robustify DNNs, it is computationally very expensive (e.g., 5-10X costlier than standard training). To address this challenge, existing approaches focus on single-step AT, referred to as Fast AT, reducing the overhead of adversarial example generation. Unfortunately, these approaches are known to fail against stronger adversaries. To make AT computationally efficient without compromising robustness, this paper takes a different view of the efficient AT problem. Specifically, we propose to minimize redundancies at the data level by leveraging data pruning. Extensive experiments demonstrate that the data pruning based AT can achieve similar or superior robust (and clean) accuracy as its unpruned counterparts while being significantly faster. For instance, proposed strategies accelerate CIFAR-10 training up to 3.44X and CIFAR-100 training to 2.02X. Additionally, the data pruning methods can readily be reconciled with existing adversarial acceleration tricks to obtain the striking speed-ups of 5.66X and 5.12X on CIFAR-10, 3.67X and 3.07X on CIFAR-100 with TRADES and MART, respectively.

翻译：深度神经网络（DNN）对对抗样本敏感，导致其在现实世界中性能脆弱且不可靠。尽管对抗训练（AT）是目前最有效的DNN鲁棒化方法之一，但其计算成本极高（例如，比标准训练昂贵5-10倍）。为解决这一挑战，现有方法聚焦于单步AT（称为Fast AT），以降低对抗样本生成的额外开销。然而，这些方法已知在面对更强攻击时会失效。为在不牺牲鲁棒性的前提下提升AT的计算效率，本文从不同视角审视高效AT问题。具体而言，我们提出通过数据剪枝来最小化数据层面的冗余。大量实验表明，基于数据剪枝的AT在实现与未剪枝变体相当或更优的鲁棒（及干净）精度的同时，显著加速训练过程。例如，所提策略将CIFAR-10训练加速高达3.44倍，CIFAR-100训练加速至2.02倍。此外，数据剪枝方法可轻松与现有对抗加速技巧结合，分别在CIFAR-10上获得5.66倍和5.12倍的显著加速，在CIFAR-100上结合TRADES和MART获得3.67倍和3.07倍的加速效果。