Stochastic variational inference offers an attractive option as a default method for differentiable probabilistic programming. However, the performance of the variational approach depends on the choice of an appropriate variational family. Here, we introduce automatic structured variational inference (ASVI), a fully automated method for constructing structured variational families, inspired by the closed-form update in conjugate Bayesian models. These convex-update families incorporate the forward pass of the input probabilistic program and can therefore capture complex statistical dependencies. Convex-update families have the same space and time complexity as the input probabilistic program and are therefore tractable for a very large family of models including both continuous and discrete variables. We validate our automatic variational method on a wide range of low- and high-dimensional inference problems. We find that ASVI provides a clear improvement in performance when compared with other popular approaches such as the mean-field approach and inverse autoregressive flows. We provide an open source implementation of ASVI in TensorFlow Probability.

翻译：托盘变异性推断提供了一种有吸引力的选择,作为不同概率编程的一种默认方法。然而,变异性方法的性能取决于选择一个适当的变式家庭。在这里,我们引入了自动结构变异性推论(ASVI),这是一种完全自动化的构建结构变式家庭的方法,这是由同源贝叶斯模型中封闭式的更新所启发的。这些共通式更新的家庭结合了输入概率程序的前传,因此可以捕捉复杂的统计依赖性。相近家庭与输入概率程序具有同样的空间和时间复杂性,因此对于包括连续变量和离散变量在内的非常庞大的模型大家庭来说,是可以移动的。我们验证了我们的自动变异性方法在一系列低度和高度推断问题上。我们发现,与中位法和反向递增性流动等其他流行方法相比,ASVI的性能有明显的改进。我们在TensorFlow 可变性中提供了ASVI的开放源实施。