Generative diffusion models apply the concept of Langevin dynamics in physics to machine leaning, attracting a lot of interest from industrial application, but a complete picture about inherent mechanisms is still lacking. In this paper, we provide a transparent physics analysis of the diffusion models, deriving the fluctuation theorem, entropy production, Franz-Parisi potential to understand the intrinsic phase transitions discovered recently. Our analysis is rooted in non-equlibrium physics and concepts from equilibrium physics, i.e., treating both forward and backward dynamics as a Langevin dynamics, and treating the reverse diffusion generative process as a statistical inference, where the time-dependent state variables serve as quenched disorder studied in spin glass theory. This unified principle is expected to guide machine learning practitioners to design better algorithms and theoretical physicists to link the machine learning to non-equilibrium thermodynamics.

翻译：生成扩散模型将物理学中的朗之万动力学概念应用于机器学习，引起了工业应用的广泛关注，但关于其内在机制的完整图景仍较为匮乏。本文从物理学角度对扩散模型进行了清晰剖析，推导出涨落定理、熵产生以及弗朗茨-帕里西势，以理解近期发现的固有相变现象。我们的分析植根于非平衡物理学与平衡态物理学概念，即将正向与反向动力学均视为朗之万动力学，并将逆向扩散生成过程视为统计推断，其中时变状态变量充当自旋玻璃理论中所研究的淬火无序。这一统一原理有望指导机器学习从业者设计更优算法，并帮助理论物理学家将机器学习与非平衡热力学联系起来。