The Gaussian process state-space models (GPSSMs) represent a versatile class of data-driven nonlinear dynamical system models. However, the presence of numerous latent variables in GPSSM incurs unresolved issues for existing variational inference approaches, particularly under the more realistic non-mean-field (NMF) assumption, including extensive training effort, compromised inference accuracy, and infeasibility for online applications, among others. In this paper, we tackle these challenges by incorporating the ensemble Kalman filter (EnKF), a well-established model-based filtering technique, into the NMF variational inference framework to approximate the posterior distribution of the latent states. This novel marriage between EnKF and GPSSM not only eliminates the need for extensive parameterization in learning variational distributions, but also enables an interpretable, closed-form approximation of the evidence lower bound (ELBO). Moreover, owing to the streamlined parameterization via the EnKF, the new GPSSM model can be easily accommodated in online learning applications. We demonstrate that the resulting EnKF-aided online algorithm embodies a principled objective function by ensuring data-fitting accuracy while incorporating model regularizations to mitigate overfitting. We also provide detailed analysis and fresh insights for the proposed algorithms. Comprehensive evaluation across diverse real and synthetic datasets corroborates the superior learning and inference performance of our EnKF-aided variational inference algorithms compared to existing methods.

翻译：高斯过程状态空间模型（GPSSM）是一类灵活的数据驱动非线性动态系统模型。然而，GPSSM中大量潜在变量的存在给现有变分推理方法带来了尚未解决的问题，尤其是在更符合实际的非平均场（NMF）假设下，这些问题包括繁重的训练负担、受损的推理精度以及在线应用不可行等。本文通过将成熟的基于模型的滤波技术——集合卡尔曼滤波（EnKF）——纳入NMF变分推理框架来近似潜在状态的后验分布，从而应对这些挑战。EnKF与GPSSM的这种新颖结合不仅消除了学习变分分布时大量参数化的需求，还实现了证据下界（ELBO）的一种可解释的闭式近似。此外，得益于通过EnKF实现的简化参数化，新的GPSSM模型能够轻松适应在线学习应用。我们证明，所得到的EnKF辅助在线算法通过确保数据拟合精度，同时结合模型正则化以减轻过拟合，体现了一种有原则的目标函数。我们还对所提出的算法进行了详细分析并提供了新的见解。在多样化的真实与合成数据集上的全面评估证实，与现有方法相比，我们的EnKF辅助变分推理算法具有更优的学习和推理性能。