Score-based generative models (SGMs) have recently emerged as a promising class of generative models. However, a fundamental limitation is that their sampling process is slow due to a need for many (\eg, $2000$) iterations of sequential computations. An intuitive acceleration method is to reduce the sampling iterations which however causes severe performance degradation. We assault this problem to the ill-conditioned issues of the Langevin dynamics and reverse diffusion in the sampling process. Under this insight, we propose a model-agnostic {\bf\em preconditioned diffusion sampling} (PDS) method that leverages matrix preconditioning to alleviate the aforementioned problem. PDS alters the sampling process of a vanilla SGM at marginal extra computation cost, and without model retraining. Theoretically, we prove that PDS preserves the output distribution of the SGM, no risk of inducing systematical bias to the original sampling process. We further theoretically reveal a relation between the parameter of PDS and the sampling iterations,easing the parameter estimation under varying sampling iterations. Extensive experiments on various image datasets with a variety of resolutions and diversity validate that our PDS consistently accelerates off-the-shelf SGMs whilst maintaining the synthesis quality. In particular, PDS can accelerate by up to $29\times$ on more challenging high resolution (1024$\times$1024) image generation. Compared with the latest generative models (\eg, CLD-SGM, DDIM, and Analytic-DDIM), PDS can achieve the best sampling quality on CIFAR-10 at a FID score of 1.99. Our code is made publicly available to foster any further research https://github.com/fudan-zvg/PDS.

翻译：分数生成模型(SGMs)近期已成为一类颇具前景的生成模型。然而，其采样过程由于需要大量(例如2000次)顺序计算迭代而存在速度慢的根本性局限。一种直观的加速方法是减少采样迭代次数，但这会导致严重的性能下降。我们将此问题归因于采样过程中郎之万动力学和反向扩散的病态条件问题。基于这一洞察，我们提出了一种模型无关的**预条件扩散采样**方法，利用矩阵预条件技术缓解上述问题。PDS以极小的额外计算成本修改原始SGM的采样过程，且无需重新训练模型。理论上，我们证明了PDS能够保持SGM的输出分布，不会对原始采样过程引入系统性偏差。此外，我们进一步从理论上揭示了PDS参数与采样迭代次数之间的关系，从而简化了不同采样迭代次数下的参数估计。在多种分辨率各异图像数据集上的大量实验证实，PDS在保持合成质量的同时，能够一致地加速现有SGM模型。特别地，在更具挑战性的高分辨率(1024×1024)图像生成任务中，PDS可实现高达29倍的加速。与最新生成模型(如CLD-SGM、DDIM和Analytic-DDIM)相比，PDS在CIFAR-10上以1.99的FID分数实现了最佳采样质量。我们的代码已在https://github.com/fudan-zvg/PDS 公开发布，以促进进一步研究。