Diffusion probabilistic models (DPMs) have exhibited excellent performance for high-fidelity image generation while suffering from inefficient sampling. Recent works accelerate the sampling procedure by proposing fast ODE solvers that leverage the specific ODE form of DPMs. However, they highly rely on specific parameterization during inference (such as noise/data prediction), which might not be the optimal choice. In this work, we propose a novel formulation towards the optimal parameterization during sampling that minimizes the first-order discretization error of the ODE solution. Based on such formulation, we propose \textit{DPM-Solver-v3}, a new fast ODE solver for DPMs by introducing several coefficients efficiently computed on the pretrained model, which we call \textit{empirical model statistics}. We further incorporate multistep methods and a predictor-corrector framework, and propose some techniques for improving sample quality at small numbers of function evaluations (NFE) or large guidance scales. Experiments show that DPM-Solver-v3 achieves consistently better or comparable performance in both unconditional and conditional sampling with both pixel-space and latent-space DPMs, especially in 5$\sim$10 NFEs. We achieve FIDs of 12.21 (5 NFE), 2.51 (10 NFE) on unconditional CIFAR10, and MSE of 0.55 (5 NFE, 7.5 guidance scale) on Stable Diffusion, bringing a speed-up of 15\%$\sim$30\% compared to previous state-of-the-art training-free methods. Code is available at \url{https://github.com/thu-ml/DPM-Solver-v3}.

翻译：扩散概率模型（DPMs）在高保真图像生成中展现出优异性能，但存在采样效率低下的问题。近期研究通过利用DPMs的特殊ODE形式提出快速ODE求解器，加速了采样过程。然而，这些方法在推理过程中高度依赖特定参数化（如噪声/数据预测），这可能并非最优选择。本文提出一种面向采样过程中最优参数化的新型公式化方法，旨在最小化ODE解的一阶离散化误差。基于该公式，我们提出了\textit{DPM-Solver-v3}——一种针对DPMs的新型快速ODE求解器，通过引入若干可在预训练模型上高效计算的系数（称为\textit{经验模型统计量}）。我们进一步融合多步方法和预测-校正框架，并提出针对低函数评估次数（NFE）或大引导尺度下改善样本质量的技术。实验表明，DPM-Solver-v3在像素空间和潜空间DPMs的无条件与条件采样中均持续取得更优或相当的性能，尤其在5$\sim$10次NFE场景下。我们在无条件CIFAR10数据集上实现了FID值12.21（5 NFE）和2.51（10 NFE），在Stable Diffusion上实现了MSE值0.55（5 NFE，7.5引导尺度），相比先前最先进的无训练方法带来15\%$\sim$30\%的加速。代码发布在\url{https://github.com/thu-ml/DPM-Solver-v3}。