Black-box variational inference is a widely-used framework for Bayesian posterior inference, but in some cases suffers from high variance in gradient estimates, harming accuracy and efficiency. This variance comes from two sources of randomness: Data subsampling and Monte Carlo sampling. Whereas existing control variates only address Monte Carlo noise and incremental gradient methods typically only address data subsampling, we propose a new "dual" control variate capable of jointly reducing variance from both sources of noise. We confirm that this leads to reduced variance and improved optimization in several real-world applications.

翻译：黑箱变分推断是一种广泛使用的贝叶斯后验推断框架，但在某些情况下会因梯度估计方差过高而影响精度与效率。该方差源于两种随机性来源：数据子采样与蒙特卡洛采样。现有控制变量仅能降低蒙特卡洛噪声，增量梯度方法通常仅处理数据子采样问题，本文提出一种新型“双重”控制变量，能够同时降低两种噪声源带来的方差。我们在多个实际应用中验证了该方法可有效降低方差并优化推断过程。