Many machine learning problems can be framed in the context of estimating functions, and often these are time-dependent functions that are estimated in real-time as observations arrive. Gaussian processes (GPs) are an attractive choice for modeling real-valued nonlinear functions due to their flexibility and uncertainty quantification. However, the typical GP regression model suffers from several drawbacks: 1) Conventional GP inference scales $O(N^{3})$ with respect to the number of observations; 2) Updating a GP model sequentially is not trivial; and 3) Covariance kernels typically enforce stationarity constraints on the function, while GPs with non-stationary covariance kernels are often intractable to use in practice. To overcome these issues, we propose a sequential Monte Carlo algorithm to fit infinite mixtures of GPs that capture non-stationary behavior while allowing for online, distributed inference. Our approach empirically improves performance over state-of-the-art methods for online GP estimation in the presence of non-stationarity in time-series data. To demonstrate the utility of our proposed online Gaussian process mixture-of-experts approach in applied settings, we show that we can sucessfully implement an optimization algorithm using online Gaussian process bandits.

翻译：许多机器学习问题可归结为函数估计问题，且这些函数常具有时间依赖性——需随观测数据实时到达进行估计。高斯过程因其灵活性与不确定性量化能力，成为建模实值非线性函数的理想选择。然而，标准高斯过程回归模型存在若干缺陷：1）传统高斯过程推断的计算复杂度随观测数量呈$O(N^{3})$增长；2）序贯更新高斯过程模型并非易事；3）协方差核通常对函数施加平稳性约束，而具有非平稳协方差核的高斯过程在实际中往往难以处理。为克服这些问题，我们提出一种序贯蒙特卡洛算法，通过拟合高斯过程无限混合模型来捕捉非平稳特性，同时支持在线分布式推断。在时序数据存在非平稳性的情况下，本方法相较现有最优在线高斯过程估计方法实现了经验性能提升。为验证所提出的在线高斯过程混合专家方法在实际场景中的效用，我们展示了其成功应用于基于在线高斯过程赌博机的优化算法实现。