Generative modeling has recently undergone remarkable advancements, primarily propelled by the transformative implications of Diffusion Probabilistic Models (DPMs). The impressive capability of these models, however, often entails significant computational overhead during both training and inference. To tackle this challenge, we present Diff-Pruning, an efficient compression method tailored for learning lightweight diffusion models from pre-existing ones, without the need for extensive re-training. The essence of Diff-Pruning is encapsulated in a Taylor expansion over pruned timesteps, a process that disregards non-contributory diffusion steps and ensembles informative gradients to identify important weights. Our empirical assessment, undertaken across four diverse datasets highlights two primary benefits of our proposed method: 1) Efficiency: it enables approximately a 50% reduction in FLOPs at a mere 10% to 20% of the original training expenditure; 2) Consistency: the pruned diffusion models inherently preserve generative behavior congruent with their pre-trained progenitors. Code is available at \url{https://github.com/VainF/Diff-Pruning}.

翻译：生成建模近期取得了显著进展，主要受扩散概率模型变革性影响的推动。然而，这些模型的强大能力通常会在训练和推理过程中带来巨大的计算开销。为应对这一挑战，我们提出了Diff-Pruning，一种高效的压缩方法，用于从预训练模型中学习轻量级扩散模型，而无需大量重新训练。Diff-Pruning的核心在于对剪枝时间步进行泰勒展开，这一过程忽略了非贡献性的扩散步骤，并集成信息性梯度以识别重要权重。我们在四个不同数据集上的实证评估突出了所提方法的两个主要优势：1）高效性：它能够在仅花费原始训练成本的10%至20%的情况下，实现约50%的FLOPs减少；2）一致性：剪枝后的扩散模型固有地保持了与预训练祖先一致的生成行为。代码可在\url{https://github.com/VainF/Diff-Pruning}获取。