Lipschitz continuity is a simple yet pivotal functional property of any predictive model that lies at the core of its robustness, generalisation, and adversarial vulnerability. Our aim is to thoroughly investigate and characterise the Lipschitz behaviour of the functions learned via neural networks. Despite the significant tightening of the bounds in the recent years, precisely estimating the Lipschitz constant continues to be a practical challenge and tight theoretical analyses, similarly, remain intractable. Therefore, we shift our perspective and instead attempt to uncover insights about the nature of Lipschitz constant of neural networks functions -- by relying on the simplest and most general upper and lower bounds. We carry out an empirical investigation in a range of different settings (architectures, losses, optimisers, label noise, etc.), which reveals several fundamental and intriguing traits of the Lipschitz continuity of neural networks functions, In particular, we identify a remarkable double descent trend in both upper and lower bounds to the Lipschitz constant which tightly aligns with the typical double descent trend in the test loss.

翻译：Lipschitz的连续性是任何预测模型的一个简单而关键的功能属性,而这种模型是其稳健性、普遍性和对抗性脆弱性的核心。我们的目标是彻底调查和描述通过神经网络所学到的功能的利普西茨行为。尽管近年来界限大大收紧,但准确估计利普西茨常数仍然是一个实际的挑战和严格的理论分析,同样,这种分析仍然是棘手的。因此,我们改变了我们的观点,而是试图通过依靠最简单和最一般的上下界来洞察利普西茨神经网络功能常数的性质。我们在不同的环境(建筑、损失、选择器、标签噪音等)中进行了实证性调查,这些调查揭示了利普西茨神经网络功能连续性的若干基本和诱人的特点,特别是,我们从上下两边都确定了显著的双位下降趋势,到利普西茨常数,这与试验损失中典型的双位趋势紧密吻合。