Fast Adversarial Training (FAT) not only improves the model robustness but also reduces the training cost of standard adversarial training. However, fast adversarial training often suffers from Catastrophic Overfitting (CO), which results in poor robustness performance. Catastrophic Overfitting describes the phenomenon of a sudden and significant decrease in robust accuracy during the training of fast adversarial training. Many effective techniques have been developed to prevent Catastrophic Overfitting and improve the model robustness from different perspectives. However, these techniques adopt inconsistent training settings and require different training costs, i.e, training time and memory costs, leading to unfair comparisons. In this paper, we conduct a comprehensive study of over 10 fast adversarial training methods in terms of adversarial robustness and training costs. We revisit the effectiveness and efficiency of fast adversarial training techniques in preventing Catastrophic Overfitting from the perspective of model local nonlinearity and propose an effective Lipschitz regularization method for fast adversarial training. Furthermore, we explore the effect of data augmentation and weight averaging in fast adversarial training and propose a simple yet effective auto weight averaging method to improve robustness further. By assembling these techniques, we propose a FGSM-based fast adversarial training method equipped with Lipschitz regularization and Auto Weight averaging, abbreviated as FGSM-LAW. Experimental evaluations on four benchmark databases demonstrate the superiority of the proposed method over state-of-the-art fast adversarial training methods and the advanced standard adversarial training methods.

翻译：快速对抗训练不仅提升了模型鲁棒性，还降低了标准对抗训练的训练成本。然而，快速对抗训练常遭受灾难性过拟合问题，导致鲁棒性能低下。灾难性过拟合描述了快速对抗训练过程中鲁棒精度突然显著下降的现象。现有研究已从不同角度开发出多种有效技术来抑制灾难性过拟合并提升模型鲁棒性。但这些技术采用不一致的训练设置，且需要不同的训练成本（即训练时间与内存开销），导致公平性比较困难。本文对超过10种快速对抗训练方法在对抗鲁棒性和训练成本方面进行了系统性研究。我们从模型局部非线性的视角重新审视了快速对抗训练技术抑制灾难性过拟合的有效性与效率，并提出了一种适用于快速对抗训练的有效Lipschitz正则化方法。此外，我们探索了数据增强和权重平均在快速对抗训练中的效果，并提出了一种简单高效的自动权重平均方法以进一步提升鲁棒性。通过整合这些技术，我们提出了一种基于FGSM的快速对抗训练方法——融合Lipschitz正则化与自动权重平均的FGSM-LAW。在四个基准数据集上的实验评估表明，该方法相较于当前最优的快速对抗训练方法及先进的对抗训练方法具有显著优越性。