Generative diffusion models have emerged as a powerful class of models in machine learning, yet a unified theoretical understanding of their operation is still developing. This paper provides an integrated perspective on generative diffusion by connecting the information-theoretic, dynamical, and thermodynamic aspects. We demonstrate that the rate of conditional entropy production during generation (i.e., the generative bandwidth) is directly governed by the expected divergence of the score function's vector field. This divergence, in turn, is linked to the branching of trajectories and generative bifurcations, which we characterize as symmetry-breaking phase transitions in the energy landscape. Beyond ensemble averages, we demonstrate that symmetry-breaking decisions are revealed by peaks in the variance of pathwise conditional entropy, capturing heterogeneity in how individual trajectories resolve uncertainty. Together, these results establish generative diffusion as a process of controlled, noise-induced symmetry breaking, in which the score function acts as a dynamic nonlinear filter that regulates both the rate and variability of information flow from noise to data.
翻译:生成扩散模型已成为机器学习中一类强大的模型,但其运作的统一理论理解仍在发展中。本文通过连接信息论、动力学和热力学方面,为生成扩散提供了整合视角。我们证明,生成过程中条件熵产生的速率(即生成带宽)直接由得分函数向量场的预期散度决定。而该散度又与轨迹分支和生成分岔相关,我们将其刻画为能量景观中的对称性破缺相变。超越系综平均层面,我们证明对称性破缺决策通过路径条件熵方差的峰值显现,捕捉了单个轨迹解决不确定性的异质性。综合来看,这些结果确立了生成扩散是一个受控的、噪声诱导的对称性破缺过程,其中得分函数充当动态非线性滤波器,调节信息从噪声流向数据的速率与变异性。