We present a novel approach for black-box VI that bypasses the difficulties of stochastic gradient ascent, including the task of selecting step-sizes. Our approach involves using a sequence of sample average approximation (SAA) problems. SAA approximates the solution of stochastic optimization problems by transforming them into deterministic ones. We use quasi-Newton methods and line search to solve each deterministic optimization problem and present a heuristic policy to automate hyperparameter selection. Our experiments show that our method simplifies the VI problem and achieves faster performance than existing methods.

翻译：我们提出一种新颖的黑箱变分推断方法，该方法绕过了随机梯度上升法的难点（包括步长选择问题）。本方法通过构建一系列样本均值近似问题来解决问题：样本均值近似通过将随机优化问题转化为确定性优化问题来逼近其解。我们采用拟牛顿法与线性搜索求解每个确定性优化问题，并设计启发式策略实现超参数自动选取。实验结果表明，本方法简化了变分推断问题，且收敛速度优于现有方法。