Hybrid models and Neural Differential Equations (NDE) are getting increasingly important for the modeling of physical systems, however they often encounter stability and accuracy issues during long-term integration. Training on unrolled trajectories is known to limit these divergences but quickly becomes too expensive due to the need for computing gradients over an iterative process. In this paper, we demonstrate that regularizing the Jacobian of the NDE model via its directional derivatives during training stabilizes long-term integration in the challenging context of short training rollouts. We design two regularizations, one for the case of known dynamics where we can directly derive the directional derivatives of the dynamic and one for the case of unknown dynamics where they are approximated using finite differences. Both methods, while having a far lower cost compared to long rollouts during training, are successful in improving the stability of long-term simulations for several ordinary and partial differential equations, opening up the door to training NDE methods for long-term integration of large scale systems.

翻译：混合模型与神经微分方程（NDE）在物理系统建模中日益重要，然而它们在长期积分过程中常遇到稳定性与精度问题。已知在展开轨迹上进行训练可限制这些发散，但由于需要在迭代过程中计算梯度，其计算成本迅速变得过高。本文证明，在训练期间通过方向导数对NDE模型的雅可比矩阵进行正则化，可在短训练展开这一挑战性背景下稳定长期积分。我们设计了两种正则化方法：一种适用于动力学已知的情况，可直接推导动力学的方向导数；另一种适用于动力学未知的情况，其中方向导数通过有限差分近似。两种方法在训练期间的计算成本远低于长展开轨迹，均成功提升了若干常微分方程与偏微分方程长期模拟的稳定性，为训练适用于大规模系统长期积分的NDE方法开辟了道路。