Sequential VAEs have been successfully considered for many high-dimensional time series modelling problems, with many variant models relying on discrete-time mechanisms such as recurrent neural networks (RNNs). On the other hand, continuous-time methods have recently gained attraction, especially in the context of irregularly-sampled time series, where they can better handle the data than discrete-time methods. One such class are Gaussian process variational autoencoders (GPVAEs), where the VAE prior is set as a Gaussian process (GP). However, a major limitation of GPVAEs is that it inherits the cubic computational cost as GPs, making it unattractive to practioners. In this work, we leverage the equivalent discrete state space representation of Markovian GPs to enable linear time GPVAE training via Kalman filtering and smoothing. For our model, Markovian GPVAE (MGPVAE), we show on a variety of high-dimensional temporal and spatiotemporal tasks that our method performs favourably compared to existing approaches whilst being computationally highly scalable.

翻译：序贯变分自编码器已成功应用于诸多高维时间序列建模问题，其中大量变体模型依赖递归神经网络等离散时间机制。另一方面，连续时间方法近年来引起广泛关注，尤其在非均匀采样时间序列场景下，此类方法相较于离散时间方法能更有效地处理数据。高斯过程变分自编码器即属此类，其先验分布被设定为高斯过程。然而，高斯过程变分自编码器的主要局限在于继承了高斯过程的立方计算复杂度，使其对实践者缺乏吸引力。本文利用马尔可夫高斯过程的等效离散状态空间表示，通过卡尔曼滤波与平滑实现线性时间复杂度的高斯过程变分自编码器训练。针对所提出的模型——马尔可夫高斯过程变分自编码器，我们在多种高维时间与时空任务上验证了该方法相较于现有方案在保持计算高度可扩展性的同时具有更优性能。