Semi-implicit variational inference (SIVI) greatly enriches the expressiveness of variational families by considering implicit variational distributions defined in a hierarchical manner. However, due to the intractable densities of variational distributions, current SIVI approaches often use surrogate evidence lower bounds (ELBOs) or employ expensive inner-loop MCMC runs for unbiased ELBOs for training. In this paper, we propose SIVI-SM, a new method for SIVI based on an alternative training objective via score matching. Leveraging the hierarchical structure of semi-implicit variational families, the score matching objective allows a minimax formulation where the intractable variational densities can be naturally handled with denoising score matching. We show that SIVI-SM closely matches the accuracy of MCMC and outperforms ELBO-based SIVI methods in a variety of Bayesian inference tasks.

翻译：半隐式变分推断（SIVI）通过考虑以层次方式定义的隐式变分分布，极大地增强了变分族的表达能力。然而，由于变分分布密度函数难以处理，当前SIVI方法通常使用代理证据下界（ELBO）或采用计算成本高昂的内循环MCMC采样来获取无偏ELBO以进行训练。本文提出SIVI-SM——一种基于分数匹配替代训练目标的新方法。利用半隐式变分族的层次结构，分数匹配目标可转化为极小极大优化形式，其中难以处理的变分密度函数可通过去噪分数匹配自然得到处理。实验表明，SIVI-SM在精度上接近MCMC方法，并在多种贝叶斯推断任务中优于基于ELBO的SIVI方法。