This paper introduces a novel family of deep dynamical models designed to represent continuous-time sequence data. This family of models generates each data point in the time series by a neural emission model, which is a non-linear transformation of a latent state vector. The trajectory of the latent states is implicitly described by a neural ordinary differential equation (ODE), with the initial state following an informative prior distribution parameterized by an energy-based model. Furthermore, we can extend this model to disentangle dynamic states from underlying static factors of variation, represented as time-invariant variables in the latent space. We train the model using maximum likelihood estimation with Markov chain Monte Carlo (MCMC) in an end-to-end manner, without requiring additional assisting components such as an inference network. Our experiments on oscillating systems, videos and real-world state sequences (MuJoCo) illustrate that ODEs with the learnable energy-based prior outperform existing counterparts, and can generalize to new dynamic parameterization, enabling long-horizon predictions.

翻译：本文提出了一种新颖的深度动态模型族，旨在表示连续时间序列数据。该模型族通过神经发射模型生成时间序列中的每个数据点，该模型是隐状态向量的非线性变换。隐状态的轨迹由神经常微分方程（ODE）隐式描述，其初始状态遵循由能量基模型参数化的信息性先验分布。此外，我们可以扩展该模型，将动态状态与潜在的静态变异因子解耦，这些因子在隐空间中表示为时不变变量。我们采用端到端的方式，通过马尔可夫链蒙特卡洛（MCMC）进行最大似然估计来训练模型，无需额外的辅助组件（如推理网络）。我们在振荡系统、视频和真实世界状态序列（MuJoCo）上的实验表明，具有可学习能量基先验的ODE优于现有同类模型，并且能够泛化到新的动态参数化设置，从而实现长时程预测。