Deep neural networks (DNNs) are known to be vulnerable to adversarial attacks. A range of defense methods have been proposed to train adversarially robust DNNs, among which adversarial training has demonstrated promising results. However, despite preliminary understandings developed for adversarial training, it is still not clear, from the architectural perspective, what configurations can lead to more robust DNNs. In this paper, we address this gap via a comprehensive investigation on the impact of network width and depth on the robustness of adversarially trained DNNs. Specifically, we make the following key observations: 1) more parameters (higher model capacity) does not necessarily help adversarial robustness; 2) reducing capacity at the last stage (the last group of blocks) of the network can actually improve adversarial robustness; and 3) under the same parameter budget, there exists an optimal architectural configuration for adversarial robustness. We also provide a theoretical analysis explaning why such network configuration can help robustness. These architectural insights can help design adversarially robust DNNs. Code is available at \url{https://github.com/HanxunH/RobustWRN}.

翻译：众所周知,深神经网络(DNNs)容易受到对抗性攻击的伤害。已经提出了一系列防御方法来培训对抗性强的DNS,其中对抗性培训显示了有希望的成果。然而,尽管为对抗性培训制定了初步谅解,但从建筑学角度看,还不清楚什么配置可以导致更强大的DNS。在本文件中,我们通过对网络宽度和深度对敌对性训练的DNS的稳健性的影响进行全面调查来弥补这一差距。具体地说,我们提出以下关键意见:1)更多的参数(高模型能力)不一定有助于对抗性强健;2)在网络的最后阶段(最后一组区块)降低能力实际上可以提高对抗性强健性;以及3)在同一参数预算下,还存在最佳的对抗性强健性建筑配置。我们还提供理论分析,说明为什么这种网络配置能够帮助稳健。这些建筑洞察可以帮助设计对抗性强的DNNNS。代码可在以下https://github.com/Hanx/RobustNRWR}查询。