To improve the robustness of deep classifiers against adversarial perturbations, many approaches have been proposed, such as designing new architectures with better robustness properties (e.g., Lipschitz-capped networks), or modifying the training process itself (e.g., min-max optimization, constrained learning, or regularization). These approaches, however, might not be effective at increasing the margin in the input (feature) space. As a result, there has been an increasing interest in developing training procedures that can directly manipulate the decision boundary in the input space. In this paper, we build upon recent developments in this category by developing a robust training algorithm whose objective is to increase the margin in the output (logit) space while regularizing the Lipschitz constant of the model along vulnerable directions. We show that these two objectives can directly promote larger margins in the input space. To this end, we develop a scalable method for calculating guaranteed differentiable upper bounds on the Lipschitz constant of neural networks accurately and efficiently. The relative accuracy of the bounds prevents excessive regularization and allows for more direct manipulation of the decision boundary. Furthermore, our Lipschitz bounding algorithm exploits the monotonicity and Lipschitz continuity of the activation layers, and the resulting bounds can be used to design new layers with controllable bounds on their Lipschitz constant. Experiments on the MNIST, CIFAR-10, and Tiny-ImageNet data sets verify that our proposed algorithm obtains competitively improved results compared to the state-of-the-art.

翻译：为提升深度分类器对抗扰动的鲁棒性，已有多种方法被提出，例如设计具有更优鲁棒性特性的新架构（如Lipschitz约束网络），或改进训练过程本身（如极小极大优化、约束学习或正则化）。然而，这些方法可能无法有效增大输入（特征）空间中的间隔。因此，开发能够直接操纵输入空间中决策边界的训练方法受到日益增长的关注。本文基于该领域的最新进展，提出一种鲁棒训练算法，其目标是在输出（逻辑值）空间中增大间隔，同时沿脆弱方向正则化模型的Lipschitz常数。我们证明这两个目标可直接促进输入空间中更大的间隔。为此，我们开发了一种可扩展的方法，用于准确高效地计算神经网络Lipschitz常数具有保证可微性的上界。该上界的相对准确性避免了过度正则化，并允许更直接地操纵决策边界。此外，我们的Lipschitz边界计算算法利用了激活层的单调性和Lipschitz连续性，所得边界可用于设计具有可控Lipschitz常数边界的新层。在MNIST、CIFAR-10和Tiny-ImageNet数据集上的实验验证表明，与现有最优方法相比，我们提出的算法获得了具有竞争力的改进结果。