Although diffusion model has shown great potential for generating higher quality images than GANs, slow sampling speed hinders its wide application in practice. Progressive distillation is thus proposed for fast sampling by progressively aligning output images of $N$-step teacher sampler with $N/2$-step student sampler. In this paper, we argue that this distillation-based accelerating method can be further improved, especially for few-step samplers, with our proposed \textbf{C}lassifier-based \textbf{F}eature \textbf{D}istillation (CFD). Instead of aligning output images, we distill teacher's sharpened feature distribution into the student with a dataset-independent classifier, making the student focus on those important features to improve performance. We also introduce a dataset-oriented loss to further optimize the model. Experiments on CIFAR-10 show the superiority of our method in achieving high quality and fast sampling. Code is provided at \url{https://github.com/zju-SWJ/RCFD}.

翻译：尽管扩散模型在生成高质量图像方面展现出超越生成对抗网络的潜力，但缓慢的采样速度阻碍了其在实际中的广泛应用。为此，渐进式蒸馏方法通过逐步对齐$N$步教师采样器与$N/2$步学生采样器的输出图像，实现了快速采样。本文提出基于分类器的特征蒸馏（CFD）方法，证明这类基于蒸馏的加速技术可得到进一步优化——特别针对少步采样器。不同于直接对齐输出图像，我们利用与数据集无关的分类器将教师模型锐化的特征分布蒸馏至学生模型，使学生模型聚焦于关键特征以提升性能。同时引入面向数据集的损失函数对模型进行额外优化。在CIFAR-10数据集上的实验表明，本方法在实现高质量输出与快速采样方面具有显著优势。代码已开源至\url{https://github.com/zju-SWJ/RCFD}。