We present a novel approach for black-box VI that bypasses the difficulties of stochastic gradient ascent, including the task of selecting step-sizes. Our approach involves using a sequence of sample average approximation (SAA) problems. SAA approximates the solution of stochastic optimization problems by transforming them into deterministic ones. We use quasi-Newton methods and line search to solve each deterministic optimization problem and present a heuristic policy to automate hyperparameter selection. Our experiments show that our method simplifies the VI problem and achieves faster performance than existing methods.

翻译：我们提出了一种新颖的黑盒变分推断方法，该方法绕过了随机梯度上升的诸多困难，包括步长选择任务。本方法通过采用一系列样本均值逼近（SAA）问题实现。SAA将随机优化问题转化为确定性优化问题来逼近其解。我们使用拟牛顿法和线性搜索求解每个确定性优化问题，并提出一种启发式策略来自动化超参数选择。实验表明，该方法简化了变分推断问题，且性能优于现有方法。