Black Box Variational Inference is a promising framework in a succession of recent efforts to make Variational Inference more ``black box". However, in basic version it either fails to converge due to instability or requires some fine-tuning of the update steps prior to execution that hinder it from being completely general purpose. We propose a method for regulating its parameter updates by reframing stochastic gradient ascent as a multivariate estimation problem. We examine the properties of the James-Stein estimator as a replacement for the arithmetic mean of Monte Carlo estimates of the gradient of the evidence lower bound. The proposed method provides relatively weaker variance reduction than Rao-Blackwellization, but offers a tradeoff of being simpler and requiring no fine tuning on the part of the analyst. Performance on benchmark datasets also demonstrate a consistent performance at par or better than the Rao-Blackwellized approach in terms of model fit and time to convergence.

翻译：黑盒变分推理是近年来一系列旨在使变分推理更加“黑盒化”的研究中一个有前景的框架。然而，其基本版本要么因不稳定性而无法收敛，要么需要在执行前对更新步骤进行一些精细调整，这阻碍了其完全通用化。我们提出了一种方法，通过将随机梯度上升重新表述为多变量估计问题来调节其参数更新。我们研究了詹姆斯-斯坦估计作为证据下界梯度蒙特卡洛估计的算术平均替代方法的特性。所提出的方法相比拉奥-布莱克韦尔化提供了相对较弱的方差减少，但作为一种权衡，它更简单且不需要分析者进行任何微调。在基准数据集上的表现也证明，在模型拟合和收敛时间方面，其性能与拉奥-布莱克韦尔化方法相当或更优。