We introduce Score identity Distillation (SiD), an innovative data-free method that distills the generative capabilities of pretrained diffusion models into a single-step generator. SiD not only facilitates an exponentially fast reduction in Fr\'echet inception distance (FID) during distillation but also approaches or even exceeds the FID performance of the original teacher diffusion models. By reformulating forward diffusion processes as semi-implicit distributions, we leverage three score-related identities to create an innovative loss mechanism. This mechanism achieves rapid FID reduction by training the generator using its own synthesized images, eliminating the need for real data or reverse-diffusion-based generation, all accomplished within significantly shortened generation time. Upon evaluation across four benchmark datasets, the SiD algorithm demonstrates high iteration efficiency during distillation and surpasses competing distillation approaches, whether they are one-step or few-step, data-free, or dependent on training data, in terms of generation quality. This achievement not only redefines the benchmarks for efficiency and effectiveness in diffusion distillation but also in the broader field of diffusion-based generation. The PyTorch implementation is available at https://github.com/mingyuanzhou/SiD

翻译：本文提出分数恒等蒸馏（SiD），这是一种创新的无数据方法，可将预训练扩散模型的生成能力蒸馏至单步生成器。SiD不仅在蒸馏过程中实现了Fr\'echet起始距离（FID）的指数级快速下降，而且达到甚至超越了原始教师扩散模型的FID性能。通过将前向扩散过程重构为半隐式分布，我们利用三个分数相关恒等式构建了创新的损失机制。该机制通过使用生成器自身合成的图像进行训练，实现了快速的FID降低，无需真实数据或基于反向扩散的生成过程，且均在显著缩短的生成时间内完成。在四个基准数据集上的评估表明，SiD算法在蒸馏过程中展现出高迭代效率，并在生成质量方面超越了现有蒸馏方法——无论是单步/少步生成、无数据方法还是依赖训练数据的方法。这一成果不仅重新定义了扩散蒸馏的效率与效能基准，也对更广泛的基于扩散的生成领域产生了深远影响。PyTorch实现已发布于https://github.com/mingyuanzhou/SiD。