We propose denoising diffusion variational inference (DDVI), a black-box variational inference algorithm for latent variable models which relies on diffusion models as flexible approximate posteriors. Specifically, our method introduces an expressive class of diffusion-based variational posteriors that perform iterative refinement in latent space; we train these posteriors with a novel regularized evidence lower bound (ELBO) on the marginal likelihood inspired by the wake-sleep algorithm. Our method is easy to implement (it fits a regularized extension of the ELBO), is compatible with black-box variational inference, and outperforms alternative classes of approximate posteriors based on normalizing flows or adversarial networks. We find that DDVI improves inference and learning in deep latent variable models across common benchmarks as well as on a motivating task in biology -- inferring latent ancestry from human genomes -- where it outperforms strong baselines on the Thousand Genomes dataset.

翻译：我们提出去噪扩散变分推断（DDVI），一种用于潜变量模型的黑盒变分推断算法，其核心在于利用扩散模型作为灵活的近似后验。具体而言，我们的方法引入了一类基于扩散的表达性变分后验，可在潜空间中进行迭代优化；我们通过一种受醒睡算法启发、针对边缘似然的新型正则化证据下界（ELBO）来训练这些后验分布。该方法易于实现（只需拟合ELBO的正则化扩展形式），兼容黑盒变分推断框架，并且在性能上优于基于归一化流或对抗网络的替代性近似后验类别。我们发现，DDVI在常见基准测试以及一项具有启发性的生物学任务——从人类基因组推断潜在祖先——中，均能提升深度潜变量模型的推断与学习性能；在千人基因组数据集上，其表现超越了现有强基线模型。