Estimating a distribution given access to its unnormalized density is pivotal in Bayesian inference, where the posterior is generally known only up to an unknown normalizing constant. Variational inference and Markov chain Monte Carlo methods are the predominant tools for this task; however, both are often challenging to apply reliably, particularly when the posterior has complex geometry. Here, we introduce Soft Contrastive Variational Inference (SoftCVI), which allows a family of variational objectives to be derived through a contrastive estimation framework. The approach parameterizes a classifier in terms of a variational distribution, reframing the inference task as a contrastive estimation problem aiming to identify a single true posterior sample among a set of samples. Despite this framing, we do not require positive or negative samples, but rather learn by sampling the variational distribution and computing ground truth soft classification labels from the unnormalized posterior itself. The objectives have zero variance gradient when the variational approximation is exact, without the need for specialized gradient estimators. We empirically investigate the performance on a variety of Bayesian inference tasks, using both simple (e.g. normal) and expressive (normalizing flow) variational distributions. We find that SoftCVI can be used to form objectives which are stable to train and mass-covering, frequently outperforming inference with other variational approaches.

翻译：给定非归一化密度函数估计其分布是贝叶斯推断中的核心问题，其中后验分布通常仅能确定到一个未知的归一化常数。变分推断与马尔可夫链蒙特卡罗方法是解决该任务的主要工具，然而两者在实际应用中常面临可靠性挑战，尤其当后验分布具有复杂几何结构时。本文提出软对比变分推断（SoftCVI），通过对比估计框架推导出一族变分目标函数。该方法将分类器参数化为变分分布的形式，将推断任务重构为对比估计问题，其目标是从一组样本中识别出唯一真实的后验样本。尽管采用此框架，我们并不需要正样本或负样本，而是通过采样变分分布并根据非归一化后验本身计算真实软分类标签进行学习。当变分近似精确时，目标函数具有零方差梯度，且无需专门的梯度估计器。我们在多种贝叶斯推断任务上实证研究了该方法的性能，分别使用简单（如正态分布）与表达能力强的（归一化流）变分分布。实验表明，SoftCVI 能够构建训练稳定且覆盖性强的目标函数，在多类任务中表现优于其他变分推断方法。