This paper provides a fresh view of the neural network (NN) pruning problem through the lens of graph theory. To achieve effective pruning, we aim to identify the main NN data flows and the corresponding NN connections that are most and least important for the performance of the full model. Unlike the standard approach to NN data flow analysis, which is based on information theory, we employ the notion of graph curvature, specifically Ollivier-Ricci curvature (ORC). ORC has been successfully used to identify important graph edges in various domains such as road traffic analysis, biological networks, and social networks. In particular, edges with negative ORC are considered bottlenecks and are therefore critical to the graph's overall connectivity, whereas positive-ORC edges are less essential. We use this intuition for NNs to (1) construct a graph induced by the NN structure and introduce the notion of neural curvature (NC) based on ORC; (2) calculate curvatures based on activation patterns for a set of input examples; and (3) demonstrate that NC can be used to rank edges according to their importance for overall NN functionality. We evaluate our method through pruning experiments on a variety of small and medium size models trained on three image datasets: MNIST, CIFAR-10, and CIFAR-100. The results indicate that our method can identify a larger number of unimportant edges compared to existing pruning methods.

翻译：本文从图论的角度为神经网络（NN）剪枝问题提供了一个全新视角。为实现有效剪枝，我们旨在识别主要的神经网络数据流及对完整模型性能最重要和最不重要的相应连接。与基于信息论的标准神经网络数据流分析不同，我们采用了图曲率的概念，特别是奥利维-里奇曲率(ORC)。ORC已被成功用于识别道路交通分析、生物网络和社交网络等多个领域中的重要图边。具体而言，具有负ORC的边被视为瓶颈，因此对图的整体连通性至关重要，而正ORC边则不太重要。我们将这一直觉应用于神经网络：（1）构建由网络结构诱导的图，并基于ORC引入神经曲率(NC)概念；（2）根据一组输入样本的激活模式计算曲率；（3）证明NC可用于根据边对整体网络功能的重要性进行排序。我们通过对在三个图像数据集（MNIST、CIFAR-10和CIFAR-100）上训练的各种中小型模型进行剪枝实验来评估该方法。结果表明，与现有剪枝方法相比，我们的方法能识别出更多不重要的边。