Distillation techniques have substantially improved the sampling speed of diffusion models, allowing of the generation within only one step or a few steps. However, these distillation methods require extensive training for each dataset, sampler, and network, which limits their practical applicability. To address this limitation, we propose a straightforward distillation approach, Distilled-ODE solvers (D-ODE solvers), that optimizes the ODE solver rather than training the denoising network. D-ODE solvers are formulated by simply applying a single parameter adjustment to existing ODE solvers. Subsequently, D-ODE solvers with smaller steps are optimized by ODE solvers with larger steps through distillation over a batch of samples. Our comprehensive experiments indicate that D-ODE solvers outperform existing ODE solvers, including DDIM, PNDM, DPM-Solver, DEIS, and EDM, especially when generating samples with fewer steps. Our method incur negligible computational overhead compared to previous distillation techniques, enabling simple and rapid integration with previous samplers. Qualitative analysis further shows that D-ODE solvers enhance image quality while preserving the sampling trajectory of ODE solvers.

翻译：蒸馏技术显著提升了扩散模型的采样速度，使其能够仅需一步或几步完成生成。然而，这些蒸馏方法需要对每个数据集、采样器和网络进行大量训练，限制了其实用性。为解决这一问题，我们提出了一种简洁的蒸馏方法——D-ODE求解器（Distilled-ODE Solvers），该方法通过优化ODE求解器而非训练去噪网络。D-ODE求解器只需对现有ODE求解器进行单一参数调整即可构建。随后，通过在一批样本上进行蒸馏，利用大步长的ODE求解器优化小步长的D-ODE求解器。我们的综合实验表明，D-ODE求解器在生成样本步数较少时，性能优于包括DDIM、PNDM、DPM-Solver、DEIS和EDM在内的现有ODE求解器。与以往蒸馏技术相比，我们的方法计算开销极小，可简单快速地集成到现有采样器中。定性分析进一步显示，D-ODE求解器在保持ODE求解器采样轨迹的同时提升了图像质量。