Diffusion models, which convert noise into new data instances by learning to reverse a Markov diffusion process, have become a cornerstone in contemporary generative modeling. While their practical power has now been widely recognized, the theoretical underpinnings remain far from mature. In this work, we develop a suite of non-asymptotic theory towards understanding the data generation process of diffusion models in discrete time, assuming access to reliable estimates of the (Stein) score functions. For a popular deterministic sampler (based on the probability flow ODE), we establish a convergence rate proportional to $1/T$ (with $T$ the total number of steps), improving upon past results; for another mainstream stochastic sampler (i.e., a type of the denoising diffusion probabilistic model (DDPM)), we derive a convergence rate proportional to $1/\sqrt{T}$, matching the state-of-the-art theory. Our theory imposes only minimal assumptions on the target data distribution (e.g., no smoothness assumption is imposed), and is developed based on an elementary yet versatile non-asymptotic approach without resorting to toolboxes for SDEs and ODEs. Further, we design two accelerated variants, improving the convergence to $1/T^2$ for the ODE-based sampler and $1/T$ for the DDPM-type sampler, which might be of independent theoretical and empirical interest.

翻译：扩散模型通过学习逆转马尔可夫扩散过程将噪声转化为新数据实例，已成为当代生成建模的基石。尽管其实际能力已获广泛认可，但其理论基础仍远未成熟。本文假设能够获取（斯坦因）得分函数的可靠估计，针对离散时间下扩散模型的数据生成过程，建立了一套非渐近理论。对于一种流行的确定性采样器（基于概率流常微分方程），我们证明了收敛速度与 $1/T$ 成正比（其中 $T$ 为总步数），优于以往结果；对于另一种主流随机采样器（即一类去噪扩散概率模型（DDPM）），我们推导出收敛速度与 $1/\sqrt{T}$ 成正比，与当前最优理论持平。本理论仅对目标数据分布施加极少的假设（例如未施加光滑性假设），并基于一种基本而通用的非渐近方法构建，无需借助随机微分方程和常微分方程的复杂工具箱。此外，我们设计了两种加速变体，将基于常微分方程采样器的收敛速度提升至 $1/T^2$，将DDPM型采样器的收敛速度提升至 $1/T$，这一结果可能具有独立的理论与实证价值。