Despite recent success, state-of-the-art learning-based models remain highly vulnerable to input changes such as adversarial examples. In order to obtain certifiable robustness against such perturbations, recent work considers Lipschitz-based regularizers or constraints while at the same time increasing prediction margin. Unfortunately, this comes at the cost of significantly decreased accuracy. In this paper, we propose a Calibrated Lipschitz-Margin Loss (CLL) that addresses this issue and improves certified robustness by tackling two problems: Firstly, commonly used margin losses do not adjust the penalties to the shrinking output distribution; caused by minimizing the Lipschitz constant $K$. Secondly, and most importantly, we observe that minimization of $K$ can lead to overly smooth decision functions. This limits the model's complexity and thus reduces accuracy. Our CLL addresses these issues by explicitly calibrating the loss w.r.t. margin and Lipschitz constant, thereby establishing full control over slack and improving robustness certificates even with larger Lipschitz constants. On CIFAR-10, CIFAR-100 and Tiny-ImageNet, our models consistently outperform losses that leave the constant unattended. On CIFAR-100 and Tiny-ImageNet, CLL improves upon state-of-the-art deterministic $L_2$ robust accuracies. In contrast to current trends, we unlock potential of much smaller models without $K=1$ constraints.

翻译：尽管近期取得了成功，最先进的基于学习的模型仍然极易受到诸如对抗样本等输入变化的影响。为了获得针对此类扰动的可认证鲁棒性，近期工作考虑采用基于Lipschitz的正则化项或约束，同时增加预测间隔。然而，这会导致准确率显著下降。本文提出了一种经校准的Lipschitz间隔损失（CLL），通过解决两个问题来改善可认证鲁棒性：首先，常用的间隔损失未能根据由最小化Lipschitz常数$K$引起的缩小输出分布来调整惩罚项；其次，更重要的是，我们观察到最小化$K$会导致决策函数过于平滑，从而限制模型复杂度并降低准确率。我们的CLL通过显式校准损失相对于间隔和Lipschitz常数的关系来解决这些问题，从而完全控制松弛度，并在使用较大Lipschitz常数时仍能提升鲁棒性证书质量。在CIFAR-10、CIFAR-100和Tiny-ImageNet数据集上，我们的模型持续优于忽略该常数的损失函数。在CIFAR-100和Tiny-ImageNet上，CLL提升了最先进的确定性$L_2$鲁棒准确率。与当前趋势相反，我们发掘了无需$K=1$约束的更小模型的潜力。