Deep equilibrium (DEQ) models have emerged as a promising class of implicit layer models, which abandon traditional depth by solving for the fixed points of a single nonlinear layer. Despite their success, the stability of the fixed points for these models remains poorly understood. By considering DEQ models as nonlinear dynamic systems, we propose a robust DEQ model named LyaDEQ with guaranteed provable stability via Lyapunov theory. The crux of our method is ensuring the Lyapunov stability of the DEQ model's fixed points, which enables the proposed model to resist minor initial perturbations. To avoid poor adversarial defense due to Lyapunov-stable fixed points being located near each other, we orthogonalize the layers after the Lyapunov stability module to separate different fixed points. We evaluate LyaDEQ models under well-known adversarial attacks, and experimental results demonstrate significant improvement in robustness. Furthermore, we show that the LyaDEQ model can be combined with other defense methods, such as adversarial training, to achieve even better adversarial robustness.

翻译：深度均衡（DEQ）模型作为一类有前景的隐式层模型，通过求解单个非线性层的不动点来摒弃传统深度结构。尽管取得了成功，但这些模型不动点的稳定性仍鲜有研究。通过将DEQ模型视为非线性动力系统，我们提出了一种名为LyaDEQ的鲁棒DEQ模型，该模型基于Lyapunov理论保证了可证明的稳定性。该方法的核心在于确保DEQ模型不动点的Lyapunov稳定性，从而使所提模型能够抵抗轻微初始扰动。为避免因Lyapunov稳定不动点彼此靠近而导致对抗防御性能下降，我们在Lyapunov稳定性模块之后对层进行正交化处理，以分离不同不动点。我们在已知对抗攻击下评估了LyaDEQ模型，实验结果表明鲁棒性显著提升。此外，我们证明LyaDEQ模型可与对抗训练等其他防御方法结合，以获得更优的对抗鲁棒性。