Neural ODEs demonstrate strong performance in generative and time-series modelling. However, training them via the adjoint method is slow compared to discrete models due to the requirement of numerically solving ODEs. To speed neural ODEs up, a common approach is to regularise the solutions. However, this approach may affect the expressivity of the model; when the trajectory itself matters, this is particularly important. In this paper, we propose an alternative way to speed up the training of neural ODEs. The key idea is to speed up the adjoint method by using Gau{\ss}-Legendre quadrature to solve integrals faster than ODE-based methods while remaining memory efficient. We also extend the idea to training SDEs using the Wong-Zakai theorem, by training a corresponding ODE and transferring the parameters. Our approach leads to faster training of neural ODEs, especially for large models. It also presents a new way to train SDE-based models.

翻译：神经ODE在生成建模和时间序列建模中表现优异。然而，与离散模型相比，通过伴随方法训练神经ODE需要数值求解常微分方程，导致训练速度较慢。为加速神经ODE训练，一种常见方法是对解进行正则化，但这种方法可能影响模型的表达能力——当轨迹本身具有重要性时尤其如此。本文提出了一种加速神经ODE训练的替代方案。其核心思想是通过高斯-勒让德求积法（Gau{\ss}-Legendre quadrature）加速伴随方法中的积分计算，在保持内存效率的同时，实现比基于ODE方法更快的积分求解速度。我们还利用Wong-Zakai定理将这一思想扩展到随机微分方程（SDE）的训练中，通过训练对应的ODE并迁移参数。本方法能有效加速神经ODE训练，尤其适用于大规模模型，并为基于SDE的模型训练提供了新路径。