In this paper, we study adversarial training on datasets that obey the long-tailed distribution, which is practical but rarely explored in previous works. Compared with conventional adversarial training on balanced datasets, this process falls into the dilemma of generating uneven adversarial examples (AEs) and an unbalanced feature embedding space, causing the resulting model to exhibit low robustness and accuracy on tail data. To combat that, we theoretically analyze the lower bound of the robust risk to train a model on a long-tailed dataset to obtain the key challenges in addressing the aforementioned dilemmas. Based on it, we propose a new adversarial training framework -- Re-balancing Adversarial Training (REAT). This framework consists of two components: (1) a new training strategy inspired by the effective number to guide the model to generate more balanced and informative AEs; (2) a carefully constructed penalty function to force a satisfactory feature space. Evaluation results on different datasets and model structures prove that REAT can effectively enhance the model's robustness and preserve the model's clean accuracy. The code can be found in https://github.com/GuanlinLee/REAT.

翻译：本文研究在服从长尾分布的数据集上进行对抗训练的问题，这在实践中具有重要性，但此前研究较少涉及。与在平衡数据集上的传统对抗训练相比，该过程面临生成不均衡对抗样本（AEs）以及特征嵌入空间不平衡的困境，导致最终模型在尾部数据上表现出较低的鲁棒性和准确率。为解决这一问题，我们从理论上分析了在长尾数据集上训练模型的鲁棒风险下界，从而确定应对上述困境的关键挑战。基于此，我们提出一种新的对抗训练框架——重平衡对抗训练（REAT）。该框架由两部分组成：（1）一种受有效样本数启发的新训练策略，用于指导模型生成更均衡且信息量更丰富的对抗样本；（2）一个精心构造的惩罚函数，用于强制获得令人满意的特征空间。在不同数据集和模型结构上的评估结果表明，REAT能有效增强模型鲁棒性并保持模型的干净准确率。代码可见于https://github.com/GuanlinLee/REAT。