Despite the promise of Lipschitz-based methods for provably-robust deep learning with deterministic guarantees, current state-of-the-art results are limited to feed-forward Convolutional Networks (ConvNets) on low-dimensional data, such as CIFAR-10. This paper investigates strategies for expanding certifiably robust training to larger, deeper models. A key challenge in certifying deep networks is efficient calculation of the Lipschitz bound for residual blocks found in ResNet and ViT architectures. We show that fast ways of bounding the Lipschitz constant for conventional ResNets are loose, and show how to address this by designing a new residual block, leading to the \emph{Linear ResNet} (LiResNet) architecture. We then introduce \emph{Efficient Margin MAximization} (EMMA), a loss function that stabilizes robust training by simultaneously penalizing worst-case adversarial examples from \emph{all} classes. Together, these contributions yield new \emph{state-of-the-art} robust accuracy on CIFAR-10/100 and Tiny-ImageNet under $\ell_2$ perturbations. Moreover, for the first time, we are able to scale up fast deterministic robustness guarantees to ImageNet, demonstrating that this approach to robust learning can be applied to real-world applications. We release our code on Github: \url{https://github.com/klasleino/gloro}.

翻译：尽管基于Lipschitz的方法在提供确定性保证的可证明鲁棒深度学习方面展现出前景，当前最先进的结果仍局限于前馈卷积网络（ConvNets）在低维数据（如CIFAR-10）上的应用。本文探索了将可认证鲁棒训练推广至更大更深模型的策略。认证深度网络的关键挑战在于高效计算ResNet和ViT架构中残差块的Lipschitz界。我们证明，传统ResNet的Lipschitz常数快速定界方法较为宽松，并提出通过设计新型残差块来解决该问题，从而得到线性ResNet（LiResNet）架构。随后，我们引入高效边界最大化（EMMA）损失函数，通过同时惩罚来自所有类别的最坏情况对抗样本，稳定鲁棒训练过程。这些贡献共同在CIFAR-10/100和Tiny-ImageNet数据集上，针对$\ell_2$扰动实现了新的最先进鲁棒准确率。此外，我们首次将快速确定性鲁棒性保证扩展至ImageNet，证明该鲁棒学习方法可应用于实际场景。相关代码已在Github上开源：\url{https://github.com/klasleino/gloro}。