In recent years, particle-based variational inference (ParVI) methods such as Stein variational gradient descent (SVGD) have grown in popularity as scalable methods for Bayesian inference. Unfortunately, the properties of such methods invariably depend on hyperparameters such as the learning rate, which must be carefully tuned by the practitioner in order to ensure convergence to the target measure at a suitable rate. In this paper, we introduce a suite of new particle-based methods for scalable Bayesian inference based on coin betting, which are entirely learning-rate free. We illustrate the performance of our approach on a range of numerical examples, including several high-dimensional models and datasets, demonstrating comparable performance to other ParVI algorithms with no need to tune a learning rate.
翻译:近年来,诸如斯坦因变分梯度下降(SVGD)等基于粒子的变分推断(ParVI)方法作为可扩展的贝叶斯推断方法日益流行。然而,此类方法的性能始终依赖学习率等超参数,实践者必须谨慎调参以确保以适当速率收敛至目标测度。本文基于抛硬币博弈,提出一套完全免学习率的全新粒子方法,用于可扩展贝叶斯推断。我们通过涵盖多个高维模型与数据集的数值示例,展示了该方法在不需调节学习率的情况下,取得了与其他ParVI算法相当的性能表现。