Recent years have witnessed significant progress in developing effective training and fast sampling techniques for diffusion models. A remarkable advancement is the use of stochastic differential equations (SDEs) and their marginal-preserving ordinary differential equations (ODEs) to describe data perturbation and generative modeling in a unified framework. In this paper, we carefully inspect the ODE-based sampling of a popular variance-exploding SDE and reveal several intriguing structures of its sampling dynamics. We discover that the data distribution and the noise distribution are smoothly connected with a quasi-linear sampling trajectory and another implicit denoising trajectory that even converges faster. Meanwhile, the denoising trajectory governs the curvature of the corresponding sampling trajectory and its various finite differences yield all second-order samplers used in practice. Furthermore, we establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm, with which we can characterize the asymptotic behavior of diffusion models and identify the empirical score deviation.

翻译：近年来，发展扩散模型的有效训练与快速采样技术取得了显著进展。一项重要突破是利用随机微分方程及其保边界的常微分方程，在统一框架下描述数据扰动与生成建模。本文深入剖析了一种流行的方差爆炸型随机微分方程的常微分方程采样方法，并揭示了其采样动态的若干有趣结构。我们发现，数据分布与噪声分布通过一条准线性采样轨迹及另一条收敛更快的隐式去噪轨迹平滑连接。同时，去噪轨迹决定了对应采样轨迹的曲率，其各种有限差分形式衍生出实践中使用的所有二阶采样器。此外，我们建立了最优常微分方程采样与经典均值漂移（模式搜索）算法之间的理论联系，据此可描述扩散模型的渐近行为，并识别经验分数偏差。