Many machine learning problems can be framed in the context of estimating functions, and often these are time-dependent functions that are estimated in real-time as observations arrive. Gaussian processes (GPs) are an attractive choice for modeling real-valued nonlinear functions due to their flexibility and uncertainty quantification. However, the typical GP regression model suffers from several drawbacks: 1) Conventional GP inference scales $O(N^{3})$ with respect to the number of observations; 2) Updating a GP model sequentially is not trivial; and 3) Covariance kernels typically enforce stationarity constraints on the function, while GPs with non-stationary covariance kernels are often intractable to use in practice. To overcome these issues, we propose a sequential Monte Carlo algorithm to fit infinite mixtures of GPs that capture non-stationary behavior while allowing for online, distributed inference. Our approach empirically improves performance over state-of-the-art methods for online GP estimation in the presence of non-stationarity in time-series data. To demonstrate the utility of our proposed online Gaussian process mixture-of-experts approach in applied settings, we show that we can sucessfully implement an optimization algorithm using online Gaussian process bandits.

翻译：许多机器学习问题可以归结为函数估计问题，且这些函数通常具有时变性，需随着观测数据的到达进行实时估计。高斯过程（GPs）因兼具灵活性与不确定性量化能力，成为建模实值非线性函数的理想选择。然而，标准GP回归模型存在以下缺陷：1）传统GP推断的计算复杂度为$O(N^{3})$（$N$为观测数据量）；2）序贯更新GP模型并非易事；3）协方差核通常对函数施加平稳性约束，而非平稳协方差核的GP在实践中往往难以处理。为克服上述问题，我们提出一种序贯蒙特卡洛算法，用于拟合无限混合高斯过程，该模型既能捕捉非平稳行为，又支持在线分布式推断。在时间序列数据存在非平稳性的场景中，我们的方法相较现有最优在线GP估计方法实现了更优的性能。为验证所提出的在线高斯过程混合专家模型在实际应用中的价值，我们展示了如何利用在线高斯过程波段算法成功实现优化任务。