Despite the promise of Lipschitz-based methods for provably-robust deep learning with deterministic guarantees, current state-of-the-art results are limited to feed-forward Convolutional Networks (ConvNets) on low-dimensional data, such as CIFAR-10. This paper investigates strategies for expanding certifiably robust training to larger, deeper models. A key challenge in certifying deep networks is efficient calculation of the Lipschitz bound for residual blocks found in ResNet and ViT architectures. We show that fast ways of bounding the Lipschitz constant for conventional ResNets are loose, and show how to address this by designing a new residual block, leading to the \emph{Linear ResNet} (LiResNet) architecture. We then introduce \emph{Efficient Margin MAximization} (EMMA), a loss function that stabilizes robust training by simultaneously penalizing worst-case adversarial examples from \emph{all} classes. Together, these contributions yield new \emph{state-of-the-art} robust accuracy on CIFAR-10/100 and Tiny-ImageNet under $\ell_2$ perturbations. Moreover, for the first time, we are able to scale up fast deterministic robustness guarantees to ImageNet, demonstrating that this approach to robust learning can be applied to real-world applications. We release our code on Github: \url{https://github.com/klasleino/gloro}.

翻译：尽管基于Lipschitz的方法在具有确定性保证的可证明鲁棒深度学习中展现出潜力，但当前最先进的结果仍局限于在低维数据（如CIFAR-10）上使用前馈卷积网络（ConvNets）。本文研究了将可认证鲁棒训练扩展到更大、更深模型的策略。在深层网络认证中，一个关键挑战在于高效计算ResNet和ViT架构中残差块的Lipschitz界。我们证明了传统ResNet的快速Lipschitz常数界定方法存在松弛性，并展示了如何通过设计新型残差块来解决这一问题，进而提出线性残差网络（LiResNet）架构。随后，我们引入高效边界最大化（EMMA）损失函数，通过同时惩罚来自所有类别的最坏情况对抗样本，稳定鲁棒训练过程。这些贡献在CIFAR-10/100和Tiny-ImageNet数据集上，针对$\ell_2$扰动实现了新的最先进鲁棒精度。更重要的是，我们首次将快速确定性鲁棒性保证扩展到ImageNet，证明这种鲁棒学习方法可应用于实际场景。我们在GitHub上发布了代码：\url{https://github.com/klasleino/gloro}。