Current structured pruning methods often result in considerable accuracy drops due to abrupt network changes and loss of information from pruned structures. To address these issues, we introduce the Decay Pruning Method (DPM), a novel smooth pruning approach with a self-rectifying mechanism. DPM consists of two key components: (i) Smooth Pruning: It converts conventional single-step pruning into multi-step smooth pruning, gradually reducing redundant structures to zero over N steps with ongoing optimization. (ii) Self-Rectifying: This procedure further enhances the aforementioned process by rectifying sub-optimal pruning based on gradient information. Our approach demonstrates strong generalizability and can be easily integrated with various existing pruning methods. We validate the effectiveness of DPM by integrating it with three popular pruning methods: OTOv2, Depgraph, and Gate Decorator. Experimental results show consistent improvements in performance compared to the original pruning methods, along with further reductions of FLOPs in most scenarios.

翻译：当前的结构化剪枝方法常因网络突变和剪除结构信息丢失而导致显著的精度下降。为解决这些问题，我们引入了衰减剪枝方法（DPM），一种具有自校正机制的新型光滑剪枝方法。DPM包含两个关键组成部分：（i）光滑剪枝：它将传统的单步剪枝转化为多步光滑剪枝，在持续优化过程中，通过N个步骤逐渐将冗余结构减少至零。（ii）自校正：该过程通过基于梯度信息校正次优剪枝，进一步增强了上述流程。我们的方法展现出强大的泛化能力，并能轻松与多种现有剪枝方法集成。我们通过将DPM与三种主流剪枝方法——OTOv2、Depgraph和Gate Decorator——相结合，验证了其有效性。实验结果表明，与原始剪枝方法相比，DPM在性能上取得了持续改进，并在大多数场景下进一步降低了FLOPs。