Pruning - that is, setting a significant subset of the parameters of a neural network to zero - is one of the most popular methods of model compression. Yet, several recent works have raised the issue that pruning may induce or exacerbate bias in the output of the compressed model. Despite existing evidence for this phenomenon, the relationship between neural network pruning and induced bias is not well-understood. In this work, we systematically investigate and characterize this phenomenon in Convolutional Neural Networks for computer vision. First, we show that it is in fact possible to obtain highly-sparse models, e.g. with less than 10% remaining weights, which do not decrease in accuracy nor substantially increase in bias when compared to dense models. At the same time, we also find that, at higher sparsities, pruned models exhibit higher uncertainty in their outputs, as well as increased correlations, which we directly link to increased bias. We propose easy-to-use criteria which, based only on the uncompressed model, establish whether bias will increase with pruning, and identify the samples most susceptible to biased predictions post-compression.

翻译：修剪——即将神经网络中大量参数置为零——是最流行的模型压缩方法之一。然而，近期多项研究提出，修剪可能引发或加剧压缩模型输出中的偏见。尽管已有证据表明这一现象的存在，但我们对其背后的机制——即神经网络修剪与诱发偏见之间的关系——仍缺乏深入理解。本研究系统性地探讨并刻画了计算机视觉领域中卷积神经网络所展现的这一现象。首先，我们证明，实际上有可能获得高度稀疏的模型（例如，剩余权重占比低于10%），这些模型在准确率上不逊色于稠密模型，且其偏见水平并不会显著增加。与此同时，我们发现在更高稀疏度下，修剪模型在输出中呈现出更高的不确定性以及增强的相关性，而这与偏见的增加之间存在直接关联。我们提出了易于使用的准则，仅需基于未压缩模型，即可判定修剪是否会加剧偏见，并识别出在模型压缩后最易受偏见影响的样本。