Graph Neural Networks (GNNs) are a powerful tool for handling structured graph data and addressing tasks such as node classification, graph classification, and clustering. However, the sparse nature of GNN computation poses new challenges for performance optimization compared to traditional deep neural networks. We address these challenges by providing a unified view of GNN computation, I/O, and memory. By analyzing the computational graphs of the Graph Convolutional Network (GCN) and Graph Attention (GAT) layers -- two widely used GNN layers -- we propose alternative computation strategies. We present adaptive operator reordering with caching, which achieves a speedup of up to 2.43x for GCN compared to the current state-of-the-art. Furthermore, an exploration of different caching schemes for GAT yields a speedup of up to 1.94x. The proposed optimizations save memory, are easily implemented across various hardware platforms, and have the potential to alleviate performance bottlenecks in training large-scale GNN models.

翻译：图神经网络（GNN）是处理结构化图数据的强大工具，可解决节点分类、图分类和聚类等任务。然而，与传统深度神经网络相比，GNN计算的稀疏性给性能优化带来了新挑战。我们通过提供GNN计算、I/O和内存的统一视角来应对这些挑战。通过分析两种广泛使用的GNN层——图卷积网络（GCN）和图注意力网络（GAT）层的计算图，我们提出了替代计算策略。我们提出了带缓存的自适应算子重排序方法，与当前最先进技术相比，GCN加速比最高达2.43倍。此外，对GAT不同缓存方案的探索实现了最高1.94倍的加速比。所提出的优化方法节省内存，易于在各种硬件平台上实现，并有望缓解大规模GNN模型训练中的性能瓶颈。