Adaptive network pruning approach has recently drawn significant attention due to its excellent capability to identify the importance and redundancy of layers and filters and customize a suitable pruning solution. However, it remains unsatisfactory since current adaptive pruning methods rely mostly on an additional monitor to score layer and filter importance, and thus faces high complexity and weak interpretability. To tackle these issues, we have deeply researched the weight reconstruction process in iterative prune-train process and propose a Protective Self-Adaptive Pruning (PSAP) method. First of all, PSAP can utilize its own information, weight sparsity ratio, to adaptively adjust pruning ratio of layers before each pruning step. Moreover, we propose a protective reconstruction mechanism to prevent important filters from being pruned through supervising gradients and to avoid unrecoverable information loss as well. Our PSAP is handy and explicit because it merely depends on weights and gradients of model itself, instead of requiring an additional monitor as in early works. Experiments on ImageNet and CIFAR-10 also demonstrate its superiority to current works in both accuracy and compression ratio, especially for compressing with a high ratio or pruning from scratch.

翻译：自适应网络剪枝方法因其能够出色地识别层和滤波器的重要性与冗余性，并定制合适的剪枝方案而受到广泛关注。然而，当前的剪枝方法仍不尽如人意，因为它们主要依赖额外的监控器来评估层和滤波器的重要性，从而面临高复杂性和弱可解释性。为解决这些问题，我们深入研究了迭代剪枝训练过程中的权重重建机制，并提出了一种保护性自适应剪枝（PSAP）方法。首先，PSAP能够利用自身的权重稀疏度信息，在每次剪枝步骤前自适应调整各层的剪枝比例。此外，我们提出了一种保护性重建机制，通过监督梯度防止重要滤波器被剪枝，并避免无法恢复的信息损失。由于PSAP仅依赖模型自身的权重和梯度，无需像早期方法那样引入额外监控器，因此简洁且可解释性强。在ImageNet和CIFAR-10上的实验表明，PSAP在准确率和压缩比上均优于当前方法，尤其是在高压缩比或从零开始剪枝的场景下。