Recent studies assessing the efficacy of pruning neural networks methods uncovered a surprising finding: when conducting ablation studies on existing pruning-at-initialization methods, namely SNIP, GraSP, SynFlow, and magnitude pruning, performances of these methods remain unchanged and sometimes even improve when randomly shuffling the mask positions within each layer (Layerwise Shuffling) or sampling new initial weight values (Reinit), while keeping pruning masks the same. We attempt to understand the reason behind such network immunity towards weight/mask modifications, by studying layer-wise statistics before and after randomization operations. We found that under each of the pruning-at-initialization methods, the distribution of unpruned weights changed minimally with randomization operations.

翻译：最近评估神经网络运行效率的研究发现一个令人惊讶的发现:当对现有的运行率初始化方法,即SNIP、GraSP、SynFlow和规模裁剪进行反动研究时,这些方法的性能保持不变,有时甚至有所改进,因为随机调整每个层内的遮罩位置(拉伊思打乱)或取样新的初始重量值(Reinit),同时保持同样的擦拭面罩。我们试图通过随机化操作前后的分层统计研究,了解这种网络的重量/表面调整豁免背后的原因。我们发现,在每次运行率初始化方法下,未调整重量的分布与随机化操作相比变化最小。