Sequential VAEs have been successfully considered for many high-dimensional time series modelling problems, with many variant models relying on discrete-time mechanisms such as recurrent neural networks (RNNs). On the other hand, continuous-time methods have recently gained attraction, especially in the context of irregularly-sampled time series, where they can better handle the data than discrete-time methods. One such class are Gaussian process variational autoencoders (GPVAEs), where the VAE prior is set as a Gaussian process (GP). However, a major limitation of GPVAEs is that it inherits the cubic computational cost as GPs, making it unattractive to practioners. In this work, we leverage the equivalent discrete state space representation of Markovian GPs to enable linear time GPVAE training via Kalman filtering and smoothing. We show on a variety of high-dimensional temporal and spatiotemporal tasks that our method performs favourably compared to existing approaches whilst being computationally highly scalable.

翻译：时序变分自编码器已成功应用于许多高维时间序列建模问题，其中众多变体模型依赖递归神经网络等离散时间机制。另一方面，连续时间方法近年来逐渐受到关注，特别是在非均匀采样时间序列场景中，此类方法相较于离散时间方法能更有效地处理数据。高斯过程变分自编码器便是其中一类方法，其先验分布被设定为高斯过程。然而，高斯过程变分自编码器的一个主要局限性在于继承了高斯过程的三次方计算复杂度，这使其对从业者缺乏吸引力。本文利用马尔可夫高斯过程的等价离散状态空间表示，通过卡尔曼滤波与平滑实现了线性时间的高斯过程变分自编码器训练。我们在多种高维时间与时空任务中证明，本方法在保持计算高度可扩展性的同时，性能优于现有方法。