We introduce Score identity Distillation (SiD), an innovative data-free method that distills the generative capabilities of pretrained diffusion models into a single-step generator. SiD not only facilitates an exponentially fast reduction in Fr\'echet inception distance (FID) during distillation but also approaches or even exceeds the FID performance of the original teacher diffusion models. By reformulating forward diffusion processes as semi-implicit distributions, we leverage three score-related identities to create an innovative loss mechanism. This mechanism achieves rapid FID reduction by training the generator using its own synthesized images, eliminating the need for real data or reverse-diffusion-based generation, all accomplished within significantly shortened generation time. Upon evaluation across four benchmark datasets, the SiD algorithm demonstrates high iteration efficiency during distillation and surpasses competing distillation approaches, whether they are one-step or few-step, data-free, or dependent on training data, in terms of generation quality. This achievement not only redefines the benchmarks for efficiency and effectiveness in diffusion distillation but also in the broader field of diffusion-based generation. The PyTorch implementation is available at https://github.com/mingyuanzhou/SiD

翻译：本文提出Score Identity Distillation（SiD），一种创新性的无数据方法，可将预训练扩散模型的生成能力蒸馏至单步生成器中。SiD不仅能在蒸馏过程中实现弗雷歇初始距离（FID）的指数级快速下降，还能逼近甚至超越原始教师扩散模型的FID性能。通过将前向扩散过程重新表述为半隐式分布，我们利用三个与分数相关的恒等式构建了创新损失机制。该机制通过使用生成器自身合成的图像进行训练，无需真实数据或基于反向扩散的生成，即可实现FID的快速降低，且生成时间显著缩短。在四个基准数据集上的评估表明，SiD算法在蒸馏过程中展现出较高的迭代效率，并在生成质量上超越了现有竞争性蒸馏方法（无论是一步/少步方法、无数据方法还是依赖训练数据的方法）。这一成就不仅重新定义了扩散蒸馏的效率与效果基准，也拓展了基于扩散的生成领域的边界。PyTorch实现代码详见https://github.com/mingyuanzhou/SiD