We construct a new kind of encoder, leveraging the expressive power of diffusion models. In a traditional variational autoencoder, the encoder and decoder jointly negotiate a latent representation of the input. This is made possible by the reparameterization trick, which simplifies training at the cost of restricting the encoder to a simple family of distributions. Replacing this encoder with a diffusion model requires rethinking how the decoder pressure can be transmitted back to the encoder, given that they tend to update their internal estimates of the latent in opposing directions. We solve this problem with an alternating training scheme, inspired by the expectation-maximization algorithm. Our method enables more reliable synchronization between encoder and decoder, while preserving the simple and efficient training objective of standard diffusion models.

翻译：我们构建了一种新型编码器，充分发挥了扩散模型的表达能力。在传统变分自编码器中，编码器与解码器共同协商构建输入的潜在表示。重参数化技巧使得这一过程成为可能，但该技巧在简化训练的同时，将编码器限制在简单的分布族内。若用扩散模型替代编码器，则需要重新思考如何将解码器的压力传回编码器——因为两者对潜在变量内部估计的更新方向往往相反。受期望最大化算法启发，我们提出交替训练方案来解决该问题。该方法能够在保持标准扩散模型简洁高效训练目标的同时，实现编码器与解码器之间更可靠的协同同步。