Diffusion models in the literature are optimized with various objectives that are special cases of a weighted loss, where the weighting function specifies the weight per noise level. Uniform weighting corresponds to maximizing the ELBO, a principled approximation of maximum likelihood. In current practice diffusion models are optimized with non-uniform weighting due to better results in terms of sample quality. In this work we expose a direct relationship between the weighted loss (with any weighting) and the ELBO objective. We show that the weighted loss can be written as a weighted integral of ELBOs, with one ELBO per noise level. If the weighting function is monotonic, then the weighted loss is a likelihood-based objective: it maximizes the ELBO under simple data augmentation, namely Gaussian noise perturbation. Our main contribution is a deeper theoretical understanding of the diffusion objective, but we also performed some experiments comparing monotonic with non-monotonic weightings, finding that monotonic weighting performs competitively with the best published results.

翻译：文献中的扩散模型通过加权损失的不同特例进行优化，其中加权函数指定每个噪声水平的权重。均匀加权对应最大化ELBO（证据下界），即一种原则性的最大似然近似方法。当前实践中，扩散模型采用非均匀加权进行优化，因为其在样本质量方面效果更佳。本研究揭示了加权损失（任意权重）与ELBO目标之间的直接联系。我们证明，加权损失可写作ELBO的加权积分，每个噪声水平对应一个ELBO。若加权函数为单调函数，则加权损失是基于似然的目标函数：它在简单数据增强（即高斯噪声扰动）下最大化ELBO。我们的主要贡献在于对扩散目标函数的理论深化理解，同时通过实验比较单调与非单调加权，发现单调加权与已发表的最优结果具有竞争力。