Despite their empirical success across a wide range of generative tasks, the fundamental principles underlying the ability of diffusion models to learn data distributions are poorly understood. In this work, we develop a new mathematical framework that explains how diffusion models can effectively learn low-dimensional distributions from a finite number of training samples without suffering from the curse of dimensionality. Specifically, motivated by the intrinsic low-dimensional structure of image data, we theoretically analyze a setting in which the data distribution is modeled as a mixture of low-rank Gaussians. Under suitable network parameterization, we show that optimizing the training objective of diffusion models is equivalent to solving the canonical subspace clustering problem over the training samples, where each subspace basis corresponds to the low-rank covariance of a Gaussian component. This equivalence allows us to show that the sample complexity for learning the underlying distribution scales linearly with the intrinsic dimension of the data, rather than exponentially with the ambient dimension. Our theoretical findings are further supported by empirical evidence that demonstrates phase transition phenomena in generalization on both synthetic and real-world image datasets. Moreover, we establish a correspondence between the learned subspace bases and semantic attributes of image data, providing a principled foundation for controllable image generation.

翻译：尽管扩散模型在各类生成任务中取得了显著的成功，但其学习数据分布能力的根本原理仍未被充分理解。本文提出了一种新的数学框架，揭示了扩散模型如何从有限训练样本中有效学习低维分布，从而避免维度灾难。具体而言，受图像数据固有低维结构的启发，我们在数据分布被建模为低秩高斯混合模型的设定下进行了理论分析。在合适的网络参数化条件下，我们证明优化扩散模型的训练目标等价于对训练样本求解标准子空间聚类问题，其中每个子空间的基对应一个高斯分量的低秩协方差。这一等价性表明，学习底层分布的样本复杂度与数据的本征维度呈线性关系，而非随环境维度呈指数增长。我们的理论发现得到了实证证据的进一步支持，在合成图像和真实世界图像数据上均观察到泛化性能的相变现象。此外，我们建立了学习到的子空间基与图像数据语义属性之间的对应关系，为可控图像生成奠定了理论基础。