Methods based on ordinary differential equations (ODEs) are widely used to build generative models of time-series. In addition to high computational overhead due to explicitly computing hidden states recurrence, existing ODE-based models fall short in learning sequence data with sharp transitions - common in many real-world systems - due to numerical challenges during optimization. In this work, we propose LS4, a generative model for sequences with latent variables evolving according to a state space ODE to increase modeling capacity. Inspired by recent deep state space models (S4), we achieve speedups by leveraging a convolutional representation of LS4 which bypasses the explicit evaluation of hidden states. We show that LS4 significantly outperforms previous continuous-time generative models in terms of marginal distribution, classification, and prediction scores on real-world datasets in the Monash Forecasting Repository, and is capable of modeling highly stochastic data with sharp temporal transitions. LS4 sets state-of-the-art for continuous-time latent generative models, with significant improvement of mean squared error and tighter variational lower bounds on irregularly-sampled datasets, while also being x100 faster than other baselines on long sequences.

翻译：基于常微分方程的方法被广泛用于构建时间序列的生成模型。由于显式计算隐状态递归带来的高计算开销，现有的基于ODE的模型在优化过程中面临数值挑战，难以学习许多现实系统中常见的具有急剧变化的时间序列数据。本文提出LS4，一种利用状态空间ODE驱动隐变量演变的序列生成模型，以提升建模能力。受近期深度状态空间模型（S4）启发，我们借助LS4的卷积表示绕过隐状态的显式计算，实现了加速。实验表明，在Monash预测库的真实世界数据集上，LS4在边际分布、分类和预测评分上显著优于以往的连续时间生成模型，并能有效建模具有剧烈时间变化的强随机数据。LS4在连续时间隐式生成模型中达到最先进水平，在不规则采样数据集上显著提升了均方误差和变分下界，同时在长序列上比基线方法快100倍。