Diffusion models have achieved remarkable progress in various domains with an intriguing ability to produce new data that do not exist in the training set. In this work, we study the hypothesis that such creativity arises from the neural network backbone learning a smoothed version of the empirical score function, which guides the denoising dynamics to generate data points that interpolate the training data. Focusing mainly on settings where the training set lies uniformly in a one-dimensional subspace, we elucidate the interplay between score smoothing and the denoising dynamics with analytical solutions and numerical experiments, demonstrating how smoothing the score function can cause the denoised data samples to interpolate the training set along the subspace. Moreover, we present theoretical and empirical evidence that learning score functions with neural networks - either with or without explicit regularization - can naturally achieve a similar effect, including when the data belong to simple nonlinear manifolds.

翻译：扩散模型在各领域取得了显著进展，其独特能力在于能够生成训练集中不存在的新数据。本研究探讨了这样一种假设：这种创造力源于神经网络主干学习到的经验分数函数的平滑版本，该平滑版本引导去噪动力学过程生成训练数据的内插点。我们主要聚焦于训练集均匀分布于一维子空间的场景，通过解析解与数值实验阐明了分数平滑与去噪动力学之间的相互作用机制，论证了分数函数平滑化如何促使去噪数据样本沿子空间对训练集进行内插。此外，我们从理论与实证两个层面证明：通过神经网络学习分数函数（无论是否采用显式正则化）均可自然实现相似效果，包括当数据位于简单非线性流形上的情形。