This paper provide several mathematical analyses of the diffusion model in machine learning. The drift term of the backwards sampling process is represented as a conditional expectation involving the data distribution and the forward diffusion. The training process aims to find such a drift function by minimizing the mean-squared residue related to the conditional expectation. Using small-time approximations of the Green's function of the forward diffusion, we show that the analytical mean drift function in DDPM and the score function in SGM asymptotically blow up in the final stages of the sampling process for singular data distributions such as those concentrated on lower-dimensional manifolds, and is therefore difficult to approximate by a network. To overcome this difficulty, we derive a new target function and associated loss, which remains bounded even for singular data distributions. We illustrate the theoretical findings with several numerical examples.

翻译：本文对机器学习中的扩散模型提供了若干数学分析。反向采样过程的漂移项被表示为涉及数据分布和前向扩散的条件期望。训练过程旨在通过最小化与该条件期望相关的均方残差来寻找该漂移函数。利用前向扩散格林函数的小时间近似，我们证明：对于集中于低维流形等奇异数据分布，DDPM中的解析平均漂移函数和SGM中的得分函数在采样过程最终阶段会渐近爆炸，因此难以被神经网络逼近。为克服这一困难，我们推导出新的目标函数及相关损失函数，该函数即使在奇异数据分布下仍保持有界。我们通过多个数值算例验证了理论发现。