Pruning is a compression method which aims to improve the efficiency of neural networks by reducing their number of parameters while maintaining a good performance, thus enhancing the performance-to-cost ratio in nontrivial ways. Of particular interest are structured pruning techniques, in which whole portions of parameters are removed altogether, resulting in easier to leverage shrunk architectures. Since its growth in popularity in the recent years, pruning gave birth to countless papers and contributions, resulting first in critical inconsistencies in the way results are compared, and then to a collective effort to establish standardized benchmarks. However, said benchmarks are based on training practices that date from several years ago and do not align with current practices. In this work, we verify how results in the recent literature of pruning hold up against networks that underwent both state-of-the-art training methods and trivial model scaling. We find that the latter clearly and utterly outperform all the literature we compared to, proving that updating standard pruning benchmarks and re-evaluating classical methods in their light is an absolute necessity. We thus introduce a new challenging baseline to compare structured pruning to: ThinResNet.

翻译：剪枝是一种压缩方法，旨在通过减少神经网络参数数量同时保持良好性能来提高效率，从而以非平凡方式提升性能成本比。结构化剪枝技术尤其值得关注，该技术整体移除参数的部分区域，从而生成更易于利用的缩小架构。近年来随着剪枝技术日益流行，催生了大量论文和成果，起初导致结果比较方式的关键性不一致，随后学术界共同努力建立标准化基准。然而，这些基准基于数年前的训练实践，与当前方法存在差异。本研究验证了近期剪枝文献中的结果在采用最先进训练方法和简单模型缩放技术的网络上的表现。我们发现后者明显且彻底优于所有对比文献，证明更新标准剪枝基准并据此重新评估经典方法势在必行。因此，我们提出了一个用于比较结构化剪枝的新挑战性基准：ThinResNet。