As real-world graphs expand in size, larger GNN models with billions of parameters are deployed. High parameter count in such models makes training and inference on graphs expensive and challenging. To reduce the computational and memory costs of GNNs, optimization methods such as pruning the redundant nodes and edges in input graphs have been commonly adopted. However, model compression, which directly targets the sparsification of model layers, has been mostly limited to traditional Deep Neural Networks (DNNs) used for tasks such as image classification and object detection. In this paper, we utilize two state-of-the-art model compression methods (1) train and prune and (2) sparse training for the sparsification of weight layers in GNNs. We evaluate and compare the efficiency of both methods in terms of accuracy, training sparsity, and training FLOPs on real-world graphs. Our experimental results show that on the ia-email, wiki-talk, and stackoverflow datasets for link prediction, sparse training with much lower training FLOPs achieves a comparable accuracy with the train and prune method. On the brain dataset for node classification, sparse training uses a lower number FLOPs (less than 1/7 FLOPs of train and prune method) and preserves a much better accuracy performance under extreme model sparsity.

翻译：随着真实世界图规模的扩大，参数达数十亿的大型GNN模型被部署应用。这类模型的高参数量使得在图上的训练与推理过程成本高昂且充满挑战。为降低GNN的计算与内存开销，通常采用剪枝输入图中冗余节点和边等优化方法。然而，直接针对模型层稀疏化的模型压缩技术，大多局限于图像分类、目标检测等任务中的传统深度神经网络（DNN）。本文利用两种前沿模型压缩方法——(1)训练后剪枝与(2)稀疏训练，实现GNN权重层的稀疏化。我们从准确率、训练稀疏度及训练FLOPs三个维度，在真实世界图上评估并对比两种方法的效率。实验结果表明：在ia-email、wiki-talk和stackoverflow数据集上的链接预测任务中，稀疏训练以显著更低的训练FLOPs实现了与训练后剪枝方法相当的准确率；在脑数据集上的节点分类任务中，稀疏训练在极端模型稀疏度下不仅使用了更少的FLOPs（不足训练后剪枝方法FLOPs的1/7），还保持了更优异的准确率性能。