The Gaussian process state-space models (GPSSMs) represent a versatile class of data-driven nonlinear dynamical system models. However, the presence of numerous latent variables in GPSSM incurs unresolved issues for existing variational inference approaches, particularly under the more realistic non-mean-field (NMF) assumption, including extensive training effort, compromised inference accuracy, and infeasibility for online applications, among others. In this paper, we tackle these challenges by incorporating the ensemble Kalman filter (EnKF), a well-established model-based filtering technique, into the NMF variational inference framework to approximate the posterior distribution of the latent states. This novel marriage between EnKF and GPSSM not only eliminates the need for extensive parameterization in learning variational distributions, but also enables an interpretable, closed-form approximation of the evidence lower bound (ELBO). Moreover, owing to the streamlined parameterization via the EnKF, the new GPSSM model can be easily accommodated in online learning applications. We demonstrate that the resulting EnKF-aided online algorithm embodies a principled objective function by ensuring data-fitting accuracy while incorporating model regularizations to mitigate overfitting. We also provide detailed analysis and fresh insights for the proposed algorithms. Comprehensive evaluation across diverse real and synthetic datasets corroborates the superior learning and inference performance of our EnKF-aided variational inference algorithms compared to existing methods.

翻译：高斯过程状态空间模型（GPSSM）是一类灵活的数据驱动非线性动力系统模型。然而，GPSSM中大量潜在变量的存在给现有变分推断方法带来了尚未解决的问题，尤其是在更符合实际的非平均场（NMF）假设下，包括训练开销大、推断精度受损以及无法适用于在线应用等。本文通过将成熟的基于模型的滤波技术——集成卡尔曼滤波（EnKF）融入NMF变分推断框架中，以近似潜在状态的后验分布，从而解决上述挑战。EnKF与GPSSM的这一新颖结合，不仅消除了学习变分分布时对大量参数化的需求，还实现了证据下界（ELBO）的可解释闭式近似。此外，得益于EnKF带来的精简参数化，新的GPSSM模型能够轻松适应在线学习应用。我们证明，所提出的EnKF辅助在线算法通过确保数据拟合精度并引入模型正则化以减轻过拟合，体现了一种原则性的目标函数。我们还对所提算法提供了详细分析和全新见解。在多样化真实与合成数据集上的全面评估证实，与现有方法相比，我们的EnKF辅助变分推断算法具有更优越的学习与推断性能。