We present variational inference with sequential sample-average approximation (VISA), a method for approximate inference in computationally intensive models, such as those based on numerical simulations. VISA extends importance-weighted forward-KL variational inference by employing a sequence of sample-average approximations, which are considered valid inside a trust region. This makes it possible to reuse model evaluations across multiple gradient steps, thereby reducing computational cost. We perform experiments on high-dimensional Gaussians, Lotka-Volterra dynamics, and a Pickover attractor, which demonstrate that VISA can achieve comparable approximation accuracy to standard importance-weighted forward-KL variational inference with computational savings of a factor two or more for conservatively chosen learning rates.

翻译：我们提出了一种基于序贯样本平均近似的变分推断方法（VISA），用于在计算密集型模型（如基于数值模拟的模型）中实现近似推断。VISA通过采用一系列在信任区域内被视为有效的样本平均近似，扩展了重要性加权前向KL变分推断。该方法使得模型评估结果可在多个梯度步骤中重复使用，从而降低计算成本。我们在高维高斯分布、Lotka-Volterra动力学模型和Pickover吸引子上进行了实验，结果表明：在保守选择学习率的情况下，VISA能够达到与标准重要性加权前向KL变分推断相当的近似精度，同时节省两倍或更多的计算量。