Diffusion models in the literature are optimized with various objectives that are special cases of a weighted loss, where the weighting function specifies the weight per noise level. Uniform weighting corresponds to maximizing the ELBO, a principled approximation of maximum likelihood. In current practice diffusion models are optimized with non-uniform weighting due to better results in terms of sample quality. In this work we expose a direct relationship between the weighted loss (with any weighting) and the ELBO objective. We show that the weighted loss can be written as a weighted integral of ELBOs, with one ELBO per noise level. If the weighting function is monotonic, then the weighted loss is a likelihood-based objective: it maximizes the ELBO under simple data augmentation, namely Gaussian noise perturbation. Our main contribution is a deeper theoretical understanding of the diffusion objective, but we also performed some experiments comparing monotonic with non-monotonic weightings, finding that monotonic weighting performs competitively with the best published results.

翻译：文献中的扩散模型通过作为加权损失特例的各种目标函数进行优化，其中加权函数指定每个噪声级别的权重。均匀加权对应最大化ELBO——最大似然的一种理论基础近似。当前实践中，扩散模型采用非均匀加权进行优化，因为其在样本质量方面表现更优。本研究揭示了加权损失（任意权重）与ELBO目标之间的直接关系。我们证明，加权损失可表示为ELBO的加权积分，每个噪声级别对应一个ELBO。若加权函数为单调函数，则加权损失是似然导向目标：它在简单数据增强（即高斯噪声扰动）下最大化ELBO。我们的主要贡献在于对扩散目标函数提供了更深刻的理论理解，同时通过对比单调与非单调加权的实验发现，单调加权可达到与最优发表结果相当的性能。