Generative modeling has recently undergone remarkable advancements, primarily propelled by the transformative implications of Diffusion Probabilistic Models (DPMs). The impressive capability of these models, however, often entails significant computational overhead during both training and inference. To tackle this challenge, we present Diff-Pruning, an efficient compression method tailored for learning lightweight diffusion models from pre-existing ones, without the need for extensive re-training. The essence of Diff-Pruning is encapsulated in a Taylor expansion over pruned timesteps, a process that disregards non-contributory diffusion steps and ensembles informative gradients to identify important weights. Our empirical assessment, undertaken across several datasets highlights two primary benefits of our proposed method: 1) Efficiency: it enables approximately a 50\% reduction in FLOPs at a mere 10\% to 20\% of the original training expenditure; 2) Consistency: the pruned diffusion models inherently preserve generative behavior congruent with their pre-trained models. Code is available at \url{https://github.com/VainF/Diff-Pruning}.

翻译：生成建模近期取得了显著进展，主要得益于扩散概率模型的变革性影响。然而，这些模型的强大能力往往伴随着训练和推理过程中巨大的计算开销。为解决这一挑战，我们提出了Diff-Pruning，一种高效的压缩方法，专门用于从已有模型中学习轻量级扩散模型，而无需大量重新训练。Diff-Pruning的核心在于对剪枝时间步进行泰勒展开，该过程忽略无贡献的扩散步骤，并通过集成信息梯度来识别重要权重。我们在多个数据集上的实证评估展示了所提方法的两个主要优势：1）高效性：在仅需原始训练开销10%至20%的情况下，实现了约50%的FLOPs减少；2）一致性：剪枝后的扩散模型本质上保留了与预训练模型一致的生成行为。代码可在\url{https://github.com/VainF/Diff-Pruning}获取。