Deep equilibrium (DEQ) models have emerged as a promising class of implicit layer models in deep learning, which abandon traditional depth by solving for the fixed points of a single nonlinear layer. Despite their success, the stability of the fixed points for these models remains poorly understood. Recently, Lyapunov theory has been applied to Neural ODEs, another type of implicit layer model, to confer adversarial robustness. By considering DEQ models as nonlinear dynamic systems, we propose a robust DEQ model named LyaDEQ with guaranteed provable stability via Lyapunov theory. The crux of our method is ensuring the fixed points of the DEQ models are Lyapunov stable, which enables the LyaDEQ models to resist the minor initial perturbations. To avoid poor adversarial defense due to Lyapunov-stable fixed points being located near each other, we add an orthogonal fully connected layer after the Lyapunov stability module to separate different fixed points. We evaluate LyaDEQ models on several widely used datasets under well-known adversarial attacks, and experimental results demonstrate significant improvement in robustness. Furthermore, we show that the LyaDEQ model can be combined with other defense methods, such as adversarial training, to achieve even better adversarial robustness.

翻译：深度均衡（DEQ）模型已成为深度学习中有前景的一类隐式层模型，这类模型通过求解单个非线性层的固定点来摒弃传统深度。尽管DEQ模型取得了成功，但其固定点的稳定性仍缺乏深入理解。近期，李雅普诺夫理论被应用于另一类隐式层模型——神经常微分方程，以赋予其对抗鲁棒性。通过将DEQ模型视为非线性动态系统，我们提出了一种名为LyaDEQ的鲁棒DEQ模型，借助李雅普诺夫理论实现可证明的稳定性保证。该方法的核心在于确保DEQ模型的固定点具有李雅普诺夫稳定性，从而使LyaDEQ模型能够抵抗微小的初始扰动。为避免因李雅普诺夫稳定固定点彼此邻近而导致对抗防御性能不佳，我们在李雅普诺夫稳定性模块之后添加一个正交全连接层以分离不同固定点。我们在多个广泛使用的数据集上，针对已知对抗攻击评估了LyaDEQ模型，实验结果表明鲁棒性显著提升。此外，我们证明LyaDEQ模型可与其他防御方法（如对抗训练）相结合，以实现更优的对抗鲁棒性。