As a widely recognized approach to deep generative modeling, Variational Auto-Encoders (VAEs) still face challenges with the quality of generated images, often presenting noticeable blurriness. This issue stems from the unrealistic assumption that approximates the conditional data distribution, $p(\textbf{x} | \textbf{z})$, as an isotropic Gaussian. In this paper, we propose a novel solution to address these issues. We illustrate how one can extract a latent space from a pre-existing diffusion model by optimizing an encoder to maximize the marginal data log-likelihood. Furthermore, we demonstrate that a decoder can be analytically derived post encoder-training, employing the Bayes rule for scores. This leads to a VAE-esque deep latent variable model, which discards the need for Gaussian assumptions on $p(\textbf{x} | \textbf{z})$ or the training of a separate decoder network. Our method, which capitalizes on the strengths of pre-trained diffusion models and equips them with latent spaces, results in a significant enhancement to the performance of VAEs.

翻译：作为深度生成建模中广泛认可的方法，变分自编码器（VAEs）在生成图像质量方面仍面临挑战，常出现明显模糊。这一问题源于其将条件数据分布 $p(\textbf{x} | \textbf{z})$ 近似为各向同性高斯分布的不切实际假设。本文提出一种新颖解决方案来应对这些问题。我们阐述了如何通过优化编码器以最大化边际数据对数似然，从预训练扩散模型中提取潜空间。进一步证明，在编码器训练后，可运用贝叶斯法则基于分数解析推导出解码器，从而构建出类VAE的深度潜变量模型。该模型无需对 $p(\textbf{x} | \textbf{z})$ 作高斯假设，也不需单独训练解码器网络。我们的方法充分发挥预训练扩散模型的优势，为其配备潜空间，显著提升了VAE的性能表现。