Acoustic Scene Classification (ASC) algorithms are usually expected to be deployed in resource-constrained systems. Existing works reduce the complexity of ASC algorithms by pruning some components, e.g. pruning channels in neural network. In practice, neural networks are often trained with sparsification such that unimportant channels can be found and further pruned. However, little efforts have been made to explore the the impact of channel sparsity on neural network pruning. To fully utilize the benefits of pruning for ASC, and to make sure the model performs consistently, we need a more profound comprehension of channel sparsification and its effects. This paper examines the internal weights acquired by convolutional neural networks that will undergone pruning. The study discusses how these weights can be utilized to create a novel metric, Weight Skewness (WS), for quantifying the sparsity of channels. We also provide a new approach to compare the performance of different pruning methods, which balances the trade-off between accuracy and complexity. The experiment results demonstrate that 1) applying higher channel sparsity to models can achieve greater compression rates while maintaining acceptable levels of accuracy; 2) the selection of pruning method has little influence on result 1); 3) MobileNets exhibit more significant benefits from channel sparsification than VGGNets and ResNets.
翻译:声学场景分类(ASC)算法通常预期部署在资源受限的系统中。现有研究通过剪枝部分组件(例如神经网络中的通道)来降低ASC算法的复杂度。实践中,神经网络常通过稀疏化训练,以识别不重要的通道并进一步剪枝。然而,关于通道稀疏性对神经网络剪枝影响的研究尚不多见。为充分利用剪枝对ASC的优势,并确保模型性能的一致性,我们需要更深入地理解通道稀疏化及其作用。本文研究了经过剪枝的卷积神经网络内部获取的权重,探讨了如何利用这些权重构建一种新型度量指标——权重偏度(WS),用于量化通道的稀疏性。我们还提供了一种新方法,通过平衡准确率与复杂度之间的权衡,比较不同剪枝方法的性能。实验结果表明:1)对模型施加更高的通道稀疏性可在保持可接受准确率的同时实现更大的压缩率;2)剪枝方法的选择对结果1)影响不大;3)MobileNets相比VGGNets和ResNets,从通道稀疏化中获得的收益更为显著。