Semi-implicit variational inference (SIVI) has been introduced to expand the analytical variational families by defining expressive semi-implicit distributions in a hierarchical manner. However, the single-layer architecture commonly used in current SIVI methods can be insufficient when the target posterior has complicated structures. In this paper, we propose hierarchical semi-implicit variational inference, called HSIVI, which generalizes SIVI to allow more expressive multi-layer construction of semi-implicit distributions. By introducing auxiliary distributions that interpolate between a simple base distribution and the target distribution, the conditional layers can be trained by progressively matching these auxiliary distributions one layer after another. Moreover, given pre-trained score networks, HSIVI can be used to accelerate the sampling process of diffusion models with the score matching objective. We show that HSIVI significantly enhances the expressiveness of SIVI on several Bayesian inference problems with complicated target distributions. When used for diffusion model acceleration, we show that HSIVI can produce high quality samples comparable to or better than the existing fast diffusion model based samplers with a small number of function evaluations on various datasets.

翻译：半隐式变分推断（SIVI）通过以分层方式定义表达性半隐式分布，扩展了分析变分族。然而，当目标后验分布具有复杂结构时，当前SIVI方法常用的单层架构可能不足。本文提出分层半隐式变分推断（HSIVI），该框架将SIVI推广至允许更具表达性的多层半隐式分布构建。通过引入在简单基分布与目标分布之间插值的辅助分布，条件层可逐层渐进匹配这些辅助分布进行训练。此外，给定预训练分数网络，HSIVI可利用分数匹配目标加速扩散模型的采样过程。我们证明，在多个具有复杂目标分布的贝叶斯推断问题中，HSIVI显著增强了SIVI的表达能力。当用于扩散模型加速时，HSIVI能在各数据集上以少量函数评估生成与现有快速扩散模型采样器相当或更优的高质量样本。