Since adversarial examples appeared and showed the catastrophic degradation they brought to DNN, many adversarial defense methods have been devised, among which adversarial training is considered the most effective. However, a recent work showed the inequality phenomena in $l_{\infty}$-adversarial training and revealed that the $l_{\infty}$-adversarially trained model is vulnerable when a few important pixels are perturbed by i.i.d. noise or occluded. In this paper, we propose a simple yet effective method called Input Gradient Distillation to release the inequality phenomena in $l_{\infty}$-adversarial training. Experiments show that while preserving the model's adversarial robustness, Input Gradient Distillation improves the model's robustness to i.i.d. noise and occlusion. Moreover, we formally explain why the equality of the model's saliency map can improve the model's robustness to i.i.d. noise or occlusion. Github:https://github.com/fhdnskfbeuv/Inuput-Gradient-Distillation

翻译：自对抗样本出现并揭示其对深度神经网络造成的灾难性性能退化以来，研究者提出了众多对抗防御方法，其中对抗训练被认为最为有效。然而，近期研究表明$l_{\infty}$对抗训练中存在不平等现象：当少量关键像素被独立同分布噪声扰动或遮挡时，经过$l_{\infty}$对抗训练的模型会变得脆弱。本文提出一种简单而有效的方法——输入梯度蒸馏（Input Gradient Distillation），用于缓解$l_{\infty}$对抗训练中的不平等现象。实验表明，输入梯度蒸馏在保持模型对抗鲁棒性的同时，提升了模型对独立同分布噪声和遮挡的鲁棒性。此外，我们形式化解释了为何模型显著性图的均衡性能提升模型对独立同分布噪声或遮挡的鲁棒性。代码地址：https://github.com/fhdnskfbeuv/Inuput-Gradient-Distillation