Deep Neural Networks (DNNs) are often over-parameterized for their tasks and can be compressed quite drastically by removing weights, a process called pruning. We investigate the impact of different pruning techniques on the classification performance and interpretability of GoogLeNet. We systematically apply unstructured and structured pruning, as well as connection sparsity (pruning of input weights) methods to the network and analyze the outcomes regarding the network's performance on the validation set of ImageNet. We also compare different retraining strategies, such as iterative pruning and one-shot pruning. We find that with sufficient retraining epochs, the performance of the networks can approximate the performance of the default GoogLeNet - and even surpass it in some cases. To assess interpretability, we employ the Mechanistic Interpretability Score (MIS) developed by Zimmermann et al. . Our experiments reveal that there is no significant relationship between interpretability and pruning rate when using MIS as a measure. Additionally, we observe that networks with extremely low accuracy can still achieve high MIS scores, suggesting that the MIS may not always align with intuitive notions of interpretability, such as understanding the basis of correct decisions.

翻译：深度神经网络（DNNs）通常对其任务存在过参数化现象，可通过移除权重进行显著压缩，这一过程称为剪枝。本研究探究了不同剪枝技术对GoogLeNet分类性能及可解释性的影响。我们系统性地应用了非结构化剪枝、结构化剪枝以及连接稀疏性（输入权重剪枝）方法，并分析了网络在ImageNet验证集上的性能表现。同时，我们比较了不同的再训练策略，如迭代剪枝与一次性剪枝。研究发现，在足够的再训练周期下，剪枝后网络的性能能够接近默认GoogLeNet的水平，在某些情况下甚至能超越原模型。为评估可解释性，我们采用了Zimmermann等人提出的机制可解释性评分（MIS）。实验结果表明，当使用MIS作为度量时，可解释性与剪枝率之间不存在显著关联。此外，我们观察到，即使准确率极低的网络仍可能获得较高的MIS分数，这表明MIS可能并不总是与直观的可解释性概念（例如理解正确决策的依据）相一致。