Deep equilibrium (DEQ) models have emerged as a promising class of implicit layer models in deep learning, which abandon traditional depth by solving for the fixed points of a single nonlinear layer. Despite their success, the stability of the fixed points for these models remains poorly understood. Recently, Lyapunov theory has been applied to Neural ODEs, another type of implicit layer model, to confer adversarial robustness. By considering DEQ models as nonlinear dynamic systems, we propose a robust DEQ model named LyaDEQ with guaranteed provable stability via Lyapunov theory. The crux of our method is ensuring the fixed points of the DEQ models are Lyapunov stable, which enables the LyaDEQ models to resist minor initial perturbations. To avoid poor adversarial defense due to Lyapunov-stable fixed points being located near each other, we add an orthogonal fully connected layer after the Lyapunov stability module to separate different fixed points. We evaluate LyaDEQ models on several widely used datasets under well-known adversarial attacks, and experimental results demonstrate significant improvement in robustness. Furthermore, we show that the LyaDEQ model can be combined with other defense methods, such as adversarial training, to achieve even better adversarial robustness.

翻译：深度均衡（DEQ）模型已成为深度学习中有前途的一类隐式层模型，它通过求解单个非线性层的固定点来摒弃传统深度。尽管取得了一定成功，但这些模型固定点的稳定性仍鲜为人知。最近，李雅普诺夫理论被应用于神经ODE（另一种隐式层模型）以赋予其对抗鲁棒性。通过将DEQ模型视为非线性动力系统，我们提出了一种名为LyaDEQ的鲁棒DEQ模型，通过李雅普诺夫理论保证了可证明的稳定性。该方法的关键在于确保DEQ模型的固定点具有李雅普诺夫稳定性，这使得LyaDEQ模型能够抵抗微小的初始扰动。为避免因李雅普诺夫稳定固定点彼此接近而导致对抗防御效果不佳，我们在李雅普诺夫稳定性模块后添加了一个正交全连接层，以分离不同固定点。我们在多个广泛使用的数据集上，针对著名的对抗攻击评估了LyaDEQ模型，实验结果表明鲁棒性显著提升。此外，我们证明LyaDEQ模型可与其他防御方法（如对抗训练）结合，实现更好的对抗鲁棒性。