We present variational inference with sequential sample-average approximation (VISA), a method for approximate inference in computationally intensive models, such as those based on numerical simulations. VISA extends importance-weighted forward-KL variational inference by employing a sequence of sample-average approximations, which are considered valid inside a trust region. This makes it possible to reuse model evaluations across multiple gradient steps, thereby reducing computational cost. We perform experiments on high-dimensional Gaussians, Lotka-Volterra dynamics, and a Pickover attractor, which demonstrate that VISA can achieve comparable approximation accuracy to standard importance-weighted forward-KL variational inference with computational savings of a factor two or more for conservatively chosen learning rates.

翻译：我们提出了基于顺序样本平均近似的变分推断方法（VISA），这是一种适用于计算密集型模型（例如基于数值模拟的模型）的近似推断方法。VISA通过采用信任域内有效的序列化样本平均近似，扩展了重要性加权前向KL散度变分推断框架。该方法允许跨多个梯度步骤复用模型评估结果，从而降低计算成本。我们在高维高斯分布、Lotka-Volterra动力学和Pickover吸引子上进行了实验，结果表明：在保守选择学习率的情况下，VISA能以两倍或更高的计算效率节省，达到与标准重要性加权前向KL散度变分推断相当的近似精度。