Diffusion probabilistic models (DPMs) have exhibited excellent performance for high-fidelity image generation while suffering from inefficient sampling. Recent works accelerate the sampling procedure by proposing fast ODE solvers that leverage the specific ODE form of DPMs. However, they highly rely on specific parameterization during inference (such as noise/data prediction), which might not be the optimal choice. In this work, we propose a novel formulation towards the optimal parameterization during sampling that minimizes the first-order discretization error of the ODE solution. Based on such formulation, we propose \textit{DPM-Solver-v3}, a new fast ODE solver for DPMs by introducing several coefficients efficiently computed on the pretrained model, which we call \textit{empirical model statistics}. We further incorporate multistep methods and a predictor-corrector framework, and propose some techniques for improving sample quality at small numbers of function evaluations (NFE) or large guidance scales. Experiments show that DPM-Solver-v3 achieves consistently better or comparable performance in both unconditional and conditional sampling with both pixel-space and latent-space DPMs, especially in 5$\sim$10 NFEs. We achieve FIDs of 12.21 (5 NFE), 2.51 (10 NFE) on unconditional CIFAR10, and MSE of 0.55 (5 NFE, 7.5 guidance scale) on Stable Diffusion, bringing a speed-up of 15\%$\sim$30\% compared to previous state-of-the-art training-free methods. Code is available at \url{https://github.com/thu-ml/DPM-Solver-v3}.

翻译：扩散概率模型在高保真图像生成中表现出色，但存在采样效率低下的问题。近期研究通过利用扩散模型的特定ODE形式提出快速ODE求解器来加速采样过程。然而，这些方法在推理过程中高度依赖特定参数化（如噪声/数据预测），这可能并非最优选择。本文提出一种面向采样过程中最优参数化的新公式，旨在最小化ODE解的一阶离散误差。基于该公式，我们提出\textit{DPM-Solver-v3}——一种用于扩散模型的新型快速ODE求解器，通过引入若干可基于预训练模型高效计算的系数（称为\textit{经验模型统计量}）。我们进一步融合多步法和预测-校正框架，并提出一系列提升小函数评估次数或大引导尺度下样本质量的技术。实验表明，DPM-Solver-v3在无条件和条件采样场景中（涵盖像素空间与潜空间扩散模型）均取得持续更优或相当的性能，尤其在5$\sim$10次函数评估条件下。在无条件CIFAR10数据集上，我们实现了12.21（5次函数评估）和2.51（10次函数评估）的FID分数；在Stable Diffusion上，以7.5引导尺度进行5次函数评估时均方误差达0.55，较此前最先进的无训练方法提速15\%$\sim$30\%。代码开源于\url{https://github.com/thu-ml/DPM-Solver-v3}。