Implicit neural networks, a.k.a., deep equilibrium networks, are a class of implicit-depth learning models where function evaluation is performed by solving a fixed point equation. They generalize classic feedforward models and are equivalent to infinite-depth weight-tied feedforward networks. While implicit models show improved accuracy and significant reduction in memory consumption, they can suffer from ill-posedness and convergence instability. This paper provides a new framework to design well-posed and robust implicit neural networks based upon contraction theory for the non-Euclidean norm $\ell_\infty$. Our framework includes (i) a novel condition for well-posedness based on one-sided Lipschitz constants, (ii) an average iteration for computing fixed-points, and (iii) explicit estimates on input-output Lipschitz constants. Additionally, we design a training problem with the well-posedness condition and the average iteration as constraints and, to achieve robust models, with the input-output Lipschitz constant as a regularizer. Our $\ell_\infty$ well-posedness condition leads to a larger polytopic training search space than existing conditions and our average iteration enjoys accelerated convergence. Finally, we perform several numerical experiments for function estimation and digit classification through the MNIST data set. Our numerical results demonstrate improved accuracy and robustness of the implicit models with smaller input-output Lipschitz bounds.

翻译：深平衡网络( a.k.a.a.) 是一组隐含的深入学习模型,其功能评价是通过解决固定点方程式进行的。这些模型将典型的向前推模型普遍化为典型的向后推模型,相当于无限深度的权重向前推网络。虽然隐含模型显示精确度提高,记忆消耗量显著减少,但可能会受到不良和趋同不稳定的影响。本文提供了一个新的框架,用以根据非欧洲的常规值($\ell ⁇ infty$)的收缩理论设计有良好和稳健的隐含神经网络。我们的框架包括:(一) 一种基于单面利普施常数的精度评估新颖的条件;(二) 计算固定点的平均重复值,以及(三) 投入-输出量利普申茨常数常数常数的清晰估计。此外,我们设计了一个培训问题,即良好保质条件和平均透支值的利普申根模型,作为固定值。我们更小的内置精确度精确度条件比我们的软度精确度精确度,通过我们的现有多盘化模型进行更精确的搜索。