The predominant success of diffusion models in generative modeling has spurred significant interest in understanding their theoretical foundations. In this work, we propose a feature learning framework aimed at analyzing and comparing the training dynamics of diffusion models with those of traditional classification models. Our theoretical analysis demonstrates that, under identical settings, diffusion models, due to the denoising objective, are encouraged to learn more balanced and comprehensive representations of the data. In contrast, neural networks with a similar architecture trained for classification tend to prioritize learning specific patterns in the data, often focusing on easy-to-learn components. To support these theoretical insights, we conduct several experiments on both synthetic and real-world datasets, which empirically validate our findings and highlight the distinct feature learning dynamics in diffusion models compared to classification.

翻译：扩散模型在生成建模领域的显著成功激发了对其理论基础的广泛研究兴趣。本文提出一个特征学习框架，旨在分析和比较扩散模型与传统分类模型的训练动态。我们的理论分析表明，在相同设置下，由于去噪目标的存在，扩散模型倾向于学习更均衡、更全面的数据表示。相比之下，具有相似架构、为分类任务训练的神经网络则倾向于优先学习数据中的特定模式，通常聚焦于易于学习的成分。为支持这些理论见解，我们在合成数据集和真实数据集上进行了多项实验，实证验证了我们的发现，并突显了扩散模型与分类模型相比在特征学习动态上的显著差异。