We propose a method for learning dynamical systems from high-dimensional empirical data that combines variational autoencoders and (spatio-)temporal attention within a framework designed to enforce certain scientifically-motivated invariances. We focus on the setting in which data are available from multiple different instances of a system whose underlying dynamical model is entirely unknown at the outset. The approach rests on a separation into an instance-specific encoding (capturing initial conditions, constants etc.) and a latent dynamics model that is itself universal across all instances/realizations of the system. The separation is achieved in an automated, data-driven manner and only empirical data are required as inputs to the model. The approach allows effective inference of system behaviour at any continuous time but does not require an explicit neural ODE formulation, which makes it efficient and highly scalable. We study behaviour through simple theoretical analyses and extensive experiments on synthetic and real-world datasets. The latter investigate learning the dynamics of complex systems based on finite data and show that the proposed approach can outperform state-of-the-art neural-dynamical models. We study also more general inductive bias in the context of transfer to data obtained under entirely novel system interventions. Overall, our results provide a promising new framework for efficiently learning dynamical models from heterogeneous data with potential applications in a wide range of fields including physics, medicine, biology and engineering.

翻译：我们提出一种从高维经验数据中学习动力学系统的方法，该方法将变分自编码器与（时空）注意力机制整合于一个旨在强化特定科学动机不变性的框架中。研究聚焦于以下场景：数据来自同一系统的多个不同实例，而该系统的底层动力学模型在初始阶段完全未知。该方法的核心是将系统分离为实例特定编码（捕捉初始条件、常数等）以及跨系统所有实例/实现通用的潜在动力学模型。该分离通过自动化数据驱动方式实现，模型仅需经验数据作为输入。该方法能有效推断任意连续时间下的系统行为，但无需显式构建神经ODE形式，从而具备高效性与强可扩展性。我们通过基础理论分析与大量合成及真实世界数据集实验研究模型行为，后者基于有限数据探究复杂系统动力学学习，并表明所提方法能超越最先进的神经动力学模型。我们还研究了在完全新型系统干预下数据迁移场景中的更泛化归纳偏置。总体而言，我们的研究结果为从异质数据中高效学习动力学模型提供了具有前景的新框架，在物理、医学、生物及工程等领域具有广泛应用潜力。