Adversarial Training (AT) has been widely proved to be an effective method to improve the adversarial robustness against adversarial examples for Deep Neural Networks (DNNs). As a variant of AT, Adversarial Robustness Distillation (ARD) has demonstrated its superior performance in improving the robustness of small student models with the guidance of large teacher models. However, both AT and ARD encounter the robust fairness problem: these models exhibit strong robustness when facing part of classes (easy class), but weak robustness when facing others (hard class). In this paper, we give an in-depth analysis of the potential factors and argue that the smoothness degree of samples' soft labels for different classes (i.e., hard class or easy class) will affect the robust fairness of DNN models from both empirical observation and theoretical analysis. Based on the above finding, we propose an Anti-Bias Soft Label Distillation (ABSLD) method to mitigate the adversarial robust fairness problem within the framework of Knowledge Distillation (KD). Specifically, ABSLD adaptively reduces the student's error risk gap between different classes to achieve fairness by adjusting the class-wise smoothness degree of samples' soft labels during the training process, and the smoothness degree of soft labels is controlled by assigning different temperatures in KD to different classes. Extensive experiments demonstrate that ABSLD outperforms state-of-the-art AT, ARD, and robust fairness methods in terms of overall performance of robustness and fairness.

翻译：对抗训练（AT）已被广泛证明是提升深度神经网络（DNN）对抗样本鲁棒性的有效方法。作为AT的一种变体，对抗鲁棒蒸馏（ARD）在大模型指导下显著提升了小模型的学生模型的鲁棒性能。然而，AT和ARD均面临鲁棒公平性问题：这些模型在处理部分类别（易类别）时展现出强鲁棒性，但面对其他类别（难类别）时鲁棒性较弱。本文从经验观察与理论分析两个层面，深入剖析了潜在影响因素，论证样本软标签在不同类别（即难类别或易类别）上的平滑程度会影响DNN模型的鲁棒公平性。基于上述发现，我们提出了一种反偏软标签蒸馏（ABSLD）方法，在知识蒸馏（KD）框架内缓解对抗鲁棒公平性问题。具体而言，ABSLD通过训练过程中自适应调整不同类别样本软标签的类别级平滑程度，降低学生模型在不同类别间的错误风险差距以实现公平性，其中软标签平滑程度由KD中为不同类别分配不同温度参数进行控制。大量实验表明，ABSLD在鲁棒性与公平性的综合性能上均优于当前最优的AT、ARD及鲁棒公平性方法。