Lipschitz constant is a fundamental property in certified robustness, as smaller values imply robustness to adversarial examples when a model is confident in its prediction. However, identifying the worst-case adversarial examples is known to be an NP-complete problem. Although over-approximation methods have shown success in neural network verification to address this challenge, reducing approximation errors remains a significant obstacle. Furthermore, these approximation errors hinder the ability to obtain tight local Lipschitz constants, which are crucial for certified robustness. Originally, grafting linearity into non-linear activation functions was proposed to reduce the number of unstable neurons, enabling scalable and complete verification. However, no prior theoretical analysis has explained how linearity grafting improves certified robustness. We instead consider linearity grafting primarily as a means of eliminating approximation errors rather than reducing the number of unstable neurons, since linear functions do not require relaxation. In this paper, we provide two theoretical contributions: 1) why linearity grafting improves certified robustness through the lens of the $l_\infty$ local Lipschitz constant, and 2) grafting linearity into non-linear activation functions, the dominant source of approximation errors, yields a tighter local Lipschitz constant. Based on these theoretical contributions, we propose a Lipschitz-aware linearity grafting method that removes dominant approximation errors, which are crucial for tightening the local Lipschitz constant, thereby improving certified robustness, even without certified training. Our extensive experiments demonstrate that grafting linearity into these influential activations tightens the $l_\infty$ local Lipschitz constant and enhances certified robustness.

翻译：Lipschitz常数是认证鲁棒性中的基本属性，因为当模型对其预测具有高置信度时，较小的Lipschitz值意味着对对抗样本的鲁棒性。然而，识别最坏情况下的对抗样本已知是一个NP完全问题。尽管过近似方法在神经网络验证中已成功应对这一挑战，但减少近似误差仍然是一个重大障碍。此外，这些近似误差阻碍了获得紧致的局部Lipschitz常数，而这对认证鲁棒性至关重要。最初，将线性性嫁接到非线性激活函数中被提出以减少不稳定神经元的数量，从而实现可扩展且完备的验证。然而，先前尚无理论分析解释线性嫁接如何提升认证鲁棒性。我们转而将线性嫁接主要视为消除近似误差的手段，而非减少不稳定神经元数量，因为线性函数无需松弛处理。本文提出两项理论贡献：1）从$l_\infty$局部Lipschitz常数的视角阐释线性嫁接提升认证鲁棒性的机理；2）将线性性嫁接到非线性激活函数（近似误差的主要来源）可产生更紧致的局部Lipschitz常数。基于这些理论贡献，我们提出一种Lipschitz感知的线性嫁接方法，该方法通过消除对收紧局部Lipschitz常数至关重要的主导性近似误差，从而提升认证鲁棒性，且无需认证训练。大量实验表明，将线性性嫁接到这些关键激活函数中能够收紧$l_\infty$局部Lipschitz常数并增强认证鲁棒性。