We propose denoising diffusion variational inference (DDVI), an approximate inference algorithm for latent variable models which relies on diffusion models as flexible variational posteriors. Specifically, our method introduces an expressive class of approximate posteriors with auxiliary latent variables that perform diffusion in latent space by reversing a user-specified noising process. We fit these models by optimizing a lower bound on the marginal likelihood inspired by the wake-sleep algorithm. Our method is easy to implement (it fits a regularized extension of the ELBO), is compatible with black-box variational inference, and outperforms alternative classes of approximate posteriors based on normalizing flows or adversarial networks. It increases the expressivity of flow-based methods via non-invertible deep recurrent architectures and avoids the instability of adversarial methods. We use DDVI on a motivating task in biology -- inferring latent ancestry from human genomes -- and we find that it outperforms strong baselines on the Thousand Genomes dataset.

翻译：我们提出去噪扩散变分推断（DDVI），一种基于扩散模型作为灵活变分后验的潜在变量模型近似推断算法。具体而言，我们的方法引入了一类具有辅助潜在变量的表达性近似后验，通过逆转用户指定的噪声过程在潜在空间中进行扩散。我们通过优化受唤醒-睡眠算法启发的边际似然下界来拟合这些模型。该方法易于实现（拟合ELBO的正则化扩展），兼容黑盒变分推断，并优于基于归一化流或对抗网络的其他近似后验类别。它通过非可逆深度循环架构增强了基于流的方法的表达能力，同时避免了对抗方法的稳定性问题。我们将DDVI应用于生物学中的一项激励性任务——从人类基因组推断潜在祖先——结果发现该方法在千人基因组数据集上优于强基线模型。