While neural networks can enjoy an outstanding flexibility and exhibit unprecedented performance, the mechanism behind their behavior is still not well-understood. To tackle this fundamental challenge, researchers have tried to restrict and manipulate some of their properties in order to gain new insights and better control on them. Especially, throughout the past few years, the concept of \emph{bi-Lipschitzness} has been proved as a beneficial inductive bias in many areas. However, due to its complexity, the design and control of bi-Lipschitz architectures are falling behind, and a model that is precisely designed for bi-Lipschitzness realizing a direct and simple control of the constants along with solid theoretical analysis is lacking. In this work, we investigate and propose a novel framework for bi-Lipschitzness that can achieve such a clear and tight control based on convex neural networks and the Legendre-Fenchel duality. Its desirable properties are illustrated with concrete experiments. We also apply this framework to uncertainty estimation and monotone problem settings to illustrate its broad range of applications.

翻译：尽管神经网络能够展现出卓越的灵活性和前所未有的性能，但其行为背后的机制仍未得到充分理解。为应对这一根本性挑战，研究人员尝试限制和调控网络的某些属性，以期获得新的见解并实现更好的控制。特别是近年来，双Lipschitz性概念已被证明在许多领域中是一种有益的归纳偏置。然而，由于其复杂性，双Lipschitz架构的设计与控制进展缓慢，目前尚缺乏一种专为双Lipschitz性设计的模型，能够实现对常数直接且简单的控制，并辅以扎实的理论分析。在本工作中，我们研究并提出了一种基于凸神经网络和Legendre-Fenchel对偶性的双Lipschitz性新框架，能够实现清晰且严格的常数控制。我们通过具体实验展示了其理想特性，并将该框架应用于不确定性估计和单调性问题场景，以说明其广泛的应用范围。