Score-based generative models have excellent performance in terms of generation quality and likelihood. They model the data distribution by matching a parameterized score network with first-order data score functions. The score network can be used to define an ODE ("score-based diffusion ODE") for exact likelihood evaluation. However, the relationship between the likelihood of the ODE and the score matching objective is unclear. In this work, we prove that matching the first-order score is not sufficient to maximize the likelihood of the ODE, by showing a gap between the maximum likelihood and score matching objectives. To fill up this gap, we show that the negative likelihood of the ODE can be bounded by controlling the first, second, and third-order score matching errors; and we further present a novel high-order denoising score matching method to enable maximum likelihood training of score-based diffusion ODEs. Our algorithm guarantees that the higher-order matching error is bounded by the training error and the lower-order errors. We empirically observe that by high-order score matching, score-based diffusion ODEs achieve better likelihood on both synthetic data and CIFAR-10, while retaining the high generation quality.

翻译：在生成质量和可能性方面,基于分数的基因化模型具有极好的性能。 它们通过匹配参数化得分网络和一阶数据评分函数来模拟数据分布。 分数网络可以用来定义精确概率评估的 ODE (“ 以核心为基础的扩散值” ) 。 但是, 以分数为基础的指数的可能性和得分匹配目标之间的关系并不明确。 在这项工作中, 我们通过显示最大概率和得分匹配目标之间的差距来证明匹配第一个分数并不足以最大限度地扩大ODE的可能性。 为了填补这一差距, 我们表明, 以一阶、 第二阶和第三阶的得分匹配错误来控制 ODE (“ 以核心为基础的扩散值 DE ” ) 的负可能性; 我们还可以提出一种新的高分分分分分比匹配方法, 以便能够对基于分的传播值进行最大可能的培训。 我们的算法保证, 更高分数的错误与培训错误和低分差错误相联。 我们从经验上观察到, 以高分比、 分法的传播指标化指标使得合成数据和CIFAR- 10 都具有较好的可能性。