Deep equilibrium (DEQ) models replace the multiple-layer stacking of conventional deep networks with a fixed-point iteration of a single-layer transformation. Having been demonstrated to be competitive in a variety of real-world scenarios, the adversarial robustness of general DEQs becomes increasingly crucial for their reliable deployment. Existing works improve the robustness of general DEQ models with the widely-used adversarial training (AT) framework, but they fail to exploit the structural uniquenesses of DEQ models. To this end, we interpret DEQs through the lens of neural dynamics and find that AT under-regulates intermediate states. Besides, the intermediate states typically provide predictions with a high prediction entropy. Informed by the correlation between the entropy of dynamical systems and their stability properties, we propose reducing prediction entropy by progressively updating inputs along the neural dynamics. During AT, we also utilize random intermediate states to compute the loss function. Our methods regulate the neural dynamics of DEQ models in this manner. Extensive experiments demonstrate that our methods substantially increase the robustness of DEQ models and even outperform the strong deep network baselines.

翻译：深度平衡（DEQ）模型用单层变换的固定点迭代替代了传统深度网络的多层堆叠。在众多实际场景中展现出竞争力的同时，通用DEQ的对抗鲁棒性对其可靠部署至关重要。现有工作采用广泛使用的对抗训练（AT）框架提升通用DEQ模型的鲁棒性，但未能利用DEQ模型的结构独特性。为此，我们从神经动力学视角解读DEQ，发现AT对中间状态的调控不足。此外，中间状态通常以高预测熵提供预测结果。基于动力系统熵与其稳定性特性的关联性，我们提出通过沿神经动力学逐步更新输入来降低预测熵。在AT过程中，我们还利用随机中间状态计算损失函数。通过这些方法，我们对DEQ模型的神经动力学进行显式调控。大量实验表明，我们的方法显著提升了DEQ模型的鲁棒性，甚至超越了强大的深度网络基线模型。