Neural ordinary differential equations (ODEs) are an emerging class of deep learning models for dynamical systems. They are particularly useful for learning an ODE vector field from observed trajectories (i.e., inverse problems). We here consider aspects of these models relevant for their application in science and engineering. Scientific predictions generally require structured uncertainty estimates. As a first contribution, we show that basic and lightweight Bayesian deep learning techniques like the Laplace approximation can be applied to neural ODEs to yield structured and meaningful uncertainty quantification. But, in the scientific domain, available information often goes beyond raw trajectories, and also includes mechanistic knowledge, e.g., in the form of conservation laws. We explore how mechanistic knowledge and uncertainty quantification interact on two recently proposed neural ODE frameworks - symplectic neural ODEs and physical models augmented with neural ODEs. In particular, uncertainty reflects the effect of mechanistic information more directly than the predictive power of the trained model could. And vice versa, structure can improve the extrapolation abilities of neural ODEs, a fact that can be best assessed in practice through uncertainty estimates. Our experimental analysis demonstrates the effectiveness of the Laplace approach on both low dimensional ODE problems and a high dimensional partial differential equation.

翻译：神经常微分方程（ODEs）是一类新兴的面向动力系统的深度学习模型。它们在学习从观测轨迹中提取ODE向量场（即逆问题）方面尤为有效。本文重点探讨这些模型在科学与工程应用中的若干关键方面。科学预测通常需要结构化的不确定性估计。作为首要贡献，我们证明轻量级基础贝叶斯深度学习技术（如拉普拉斯近似）可应用于神经ODE，以生成具有结构意义的不确定性量化结果。然而在科学领域，可用信息往往超越原始轨迹，还包含机制性知识（如守恒定律）。我们研究了机制性知识与不确定性量化在两类最新提出的神经ODE框架——辛神经ODE和基于神经ODE增强的物理模型——中的相互作用。具体而言，不确定性比训练模型的预测能力更直接地反映机制性信息的影响；反之，结构可提升神经ODE的外推能力，而这一特性在实践中需要依靠不确定性估计来最佳评估。我们的实验分析表明，拉普拉斯方法在低维ODE问题和高维偏微分方程中均表现出有效性。