Neural Ordinary Differential Equations (NODEs) probed the usage of numerical solvers to solve the differential equation characterized by a Neural Network (NN), therefore initiating a new paradigm of deep learning models with infinite depth. NODEs were designed to tackle the irregular time series problem. However, NODEs have demonstrated robustness against various noises and adversarial attacks. This paper is about the natural robustness of NODEs and examines the cause behind such surprising behaviour. We show that by controlling the Lipschitz constant of the ODE dynamics the robustness can be significantly improved. We derive our approach from Grownwall's inequality. Further, we draw parallels between contractivity theory and Grownwall's inequality. Experimentally we corroborate the enhanced robustness on numerous datasets - MNIST, CIFAR-10, and CIFAR 100. We also present the impact of adaptive and non-adaptive solvers on the robustness of NODEs.

翻译：神经常微分方程（NODEs）开创了利用数值求解器求解由神经网络（NN）表征的微分方程的方法，从而引入了一种具有无限深度的深度学习模型新范式。NODEs最初旨在处理不规则时间序列问题，然而研究表明其具备对多种噪声及对抗攻击的鲁棒性。本文聚焦于NODEs的天然鲁棒性特性，并剖析这一意外表现的内在成因。我们证明，通过控制ODE动力学的Lipschitz常数，可显著提升鲁棒性。该方法基于Gronwall不等式推导得出，并进一步揭示了收缩理论与Gronwall不等式之间的关联。实验方面，我们在MNIST、CIFAR-10和CIFAR-100等多个数据集上验证了增强后的鲁棒性，同时探讨了自适应与非自适应求解器对NODEs鲁棒性的影响。