Diffusion models in the literature are optimized with various objectives that are special cases of a weighted loss, where the weighting function specifies the weight per noise level. Uniform weighting corresponds to maximizing the ELBO, a principled approximation of maximum likelihood. In current practice diffusion models are optimized with non-uniform weighting due to better results in terms of sample quality. In this work we expose a direct relationship between the weighted loss (with any weighting) and the ELBO objective. We show that the weighted loss can be written as a weighted integral of ELBOs, with one ELBO per noise level. If the weighting function is monotonic, then the weighted loss is a likelihood-based objective: it maximizes the ELBO under simple data augmentation, namely Gaussian noise perturbation. Our main contribution is a deeper theoretical understanding of the diffusion objective, but we also performed some experiments comparing monotonic with non-monotonic weightings, finding that monotonic weighting performs competitively with the best published results.

翻译：文献中的扩散模型通过多种目标函数进行优化，这些目标函数是加权损失的特例，其中加权函数指定了每个噪声水平的权重。均匀加权对应于最大化ELBO（证据下界），这是最大似然的一种有原则的近似。在当前的实践中，扩散模型采用非均匀加权进行优化，因其在样本质量方面取得了更优的结果。本文揭示了加权损失（具有任意权重）与ELBO目标之间的直接关系。我们证明，加权损失可以表示为ELBO的加权积分，每个噪声水平对应一个ELBO。如果加权函数是单调的，那么加权损失是一种基于似然的目标：它通过简单数据增强（即高斯噪声扰动）来最大化ELBO。我们的主要贡献在于对扩散目标提供了更深层次的理论理解。我们还进行了实验，比较了单调加权与非单调加权，发现单调加权与已公布的最佳结果相比具有竞争力。