Variational Inference (VI) is a fundamental inference technique in Bayesian machine learning for approximating complex posterior distributions. Traditional VI often relies on the mean-field factorization, which can inadequately capture true posterior complexity. Recent advancements have leveraged neural networks to model implicit distributions, offering increased flexibility. However, the practical constraints of neural network architectures still produces inaccuracies. In this paper, we propose a method called Implicit Variational Rejection Sampling (IVRS), which integrates implicit distributions with rejection sampling to improve the posterior approximation. Our method uses neural networks to construct implicit proposal distributions, and rejection sampling with a discriminator network that estimates the density ratio between the implicit proposal and the true posterior for refining the approximation. Towards this end, we introduce the Implicit Resampling Evidence Lower Bound (IR-ELBO) as a metric to characterize the resampled distribution's quality and derive a tighter variational lower bound. Experimental results demonstrate that our method outperforms traditional variational inference techniques.

翻译：变分推断（VI）是贝叶斯机器学习中用于近似复杂后验分布的基本推断技术。传统变分推断通常依赖平均场分解，这难以充分捕捉真实后验的复杂性。近期的进展利用神经网络建模隐式分布，提供了更高的灵活性。然而，神经网络架构的实际约束仍会导致近似误差。本文提出一种名为隐式变分拒绝采样（Implicit Variational Rejection Sampling, IVRS）的方法，该方法将隐式分布与拒绝采样相结合，以改进后验近似。我们利用神经网络构建隐式提议分布，并通过带有判别网络的拒绝采样来优化近似：该判别网络用于估计隐式提议分布与真实后验之间的密度比。为此，我们引入隐式重采样证据下界（IR-ELBO）作为衡量重采样分布质量的指标，并推导出更紧致的变分下界。实验结果表明，我们的方法优于传统变分推断技术。