Generative modeling has recently undergone remarkable advancements, primarily propelled by the transformative implications of Diffusion Probabilistic Models (DPMs). The impressive capability of these models, however, often entails significant computational overhead during both training and inference. To tackle this challenge, we present Diff-Pruning, an efficient compression method tailored for learning lightweight diffusion models from pre-existing ones, without the need for extensive re-training. The essence of Diff-Pruning is encapsulated in a Taylor expansion over pruned timesteps, a process that disregards non-contributory diffusion steps and ensembles informative gradients to identify important weights. Our empirical assessment, undertaken across several datasets highlights two primary benefits of our proposed method: 1) Efficiency: it enables approximately a 50\% reduction in FLOPs at a mere 10\% to 20\% of the original training expenditure; 2) Consistency: the pruned diffusion models inherently preserve generative behavior congruent with their pre-trained models. Code is available at \url{https://github.com/VainF/Diff-Pruning}.

翻译：近期，生成式建模取得了显著进展，这主要得益于扩散概率模型（DPMs）的变革性影响。然而，这些模型的强大能力通常伴随着训练和推理过程中巨大的计算开销。为应对这一挑战，我们提出了Diff-Pruning，这是一种高效的压缩方法，专门用于从已有模型中学习轻量级扩散模型，而无需大量重新训练。Diff-Pruning的核心在于对剪枝时间步进行泰勒展开，该过程忽略非贡献的扩散步骤，并集成信息丰富的梯度来识别重要权重。我们在多个数据集上进行的实证评估展示了所提方法的两个主要优势：1）高效性：在仅需原始训练开销10%至20%的情况下，可实现约50%的FLOPs缩减；2）一致性：剪枝后的扩散模型固有地保持了与预训练模型一致的生成行为。代码可在\url{https://github.com/VainF/Diff-Pruning}获取。