While neural networks can enjoy an outstanding flexibility and exhibit unprecedented performance, the mechanism behind their behavior is still not well-understood. To tackle this fundamental challenge, researchers have tried to restrict and manipulate some of their properties in order to gain new insights and better control on them. Especially, throughout the past few years, the concept of \emph{bi-Lipschitzness} has been proved as a beneficial inductive bias in many areas. However, due to its complexity, the design and control of bi-Lipschitz architectures are falling behind, and a model that is precisely designed for bi-Lipschitzness realizing a direct and simple control of the constants along with solid theoretical analysis is lacking. In this work, we investigate and propose a novel framework for bi-Lipschitzness that can achieve such a clear and tight control based on convex neural networks and the Legendre-Fenchel duality. Its desirable properties are illustrated with concrete experiments. We also apply this framework to uncertainty estimation and monotone problem settings to illustrate its broad range of applications.

翻译：尽管神经网络具有卓越的灵活性和前所未有的性能表现，但其行为背后的机制仍未得到充分理解。为应对这一根本性挑战，研究人员尝试通过限制和调控神经网络的某些特性来获得新的见解并实现更好的控制。特别是在过去几年中，\emph{双利普希茨性} 的概念已被证明在诸多领域是一种有益的归纳偏置。然而，由于其复杂性，双利普希茨架构的设计与控制相对滞后，目前尚缺乏一种专门为双利普希茨性精确设计、能直接简洁地控制相关常数并具备坚实理论分析的模型。本工作基于凸神经网络和勒让德-芬切尔对偶性，研究并提出了一种新型双利普希茨性框架，该框架能够实现清晰且严格的控制。我们通过具体实验展示了其优良特性，并将该框架应用于不确定性估计和单调问题设定中，以说明其广泛的应用前景。