Practical Bayesian learning often requires (1) online inference, (2) dynamic models, and (3) ensembling over multiple different models. Recent advances have shown how to use random feature approximations to achieve scalable, online ensembling of Gaussian processes with desirable theoretical properties and fruitful applications. One key to these methods' success is the inclusion of a random walk on the model parameters, which makes models dynamic. We show that these methods can be generalized easily to any basis expansion model and that using alternative basis expansions, such as Hilbert space Gaussian processes, often results in better performance. To simplify the process of choosing a specific basis expansion, our method's generality also allows the ensembling of several entirely different models, for example, a Gaussian process and polynomial regression. Finally, we propose a novel method to ensemble static and dynamic models together.

翻译：实用贝叶斯学习通常需要：(1)在线推断，(2)动态模型，以及(3)对多个不同模型进行集成。近期进展展示了如何利用随机特征近似实现高斯过程的可扩展在线集成，该方法具有理想的理论性质与丰富的应用场景。这些方法成功的关键在于对模型参数引入随机游走机制，从而使模型具备动态性。我们证明此类方法可轻松推广至任意基扩展模型，且采用替代性基扩展（如希尔伯特空间高斯过程）往往能获得更优性能。为简化特定基扩展的选择过程，本方法的通用性还允许集成多个完全不同的模型，例如高斯过程与多项式回归。最后，我们提出了一种集成静态模型与动态模型的新方法。