Large-scale graph training is a notoriously challenging problem for graph neural networks (GNNs). Due to the nature of evolving graph structures into the training process, vanilla GNNs usually fail to scale up, limited by the GPU memory space. Up to now, though numerous scalable GNN architectures have been proposed, we still lack a comprehensive survey and fair benchmark of this reservoir to find the rationale for designing scalable GNNs. To this end, we first systematically formulate the representative methods of large-scale graph training into several branches and further establish a fair and consistent benchmark for them by a greedy hyperparameter searching. In addition, regarding efficiency, we theoretically evaluate the time and space complexity of various branches and empirically compare them w.r.t GPU memory usage, throughput, and convergence. Furthermore, We analyze the pros and cons for various branches of scalable GNNs and then present a new ensembling training manner, named EnGCN, to address the existing issues. Our code is available at https://github.com/VITA-Group/Large_Scale_GCN_Benchmarking.

翻译：大规模图训练对图神经网络（GNNs）而言是一个极具挑战性的难题。由于图结构在训练过程中不断演化的特性，原始GNNs通常受限于GPU内存空间而难以实现扩展。迄今为止，尽管已有众多可扩展的GNN架构被提出，但我们仍缺乏对这一领域成果的全面综述和公平基准测试，以探寻设计可扩展GNNs的内在原理。为此，我们首先将大规模图训练的代表性方法系统性地划分为若干分支，并通过贪婪超参数搜索为它们建立了公平且一致的基准测试。此外，在效率方面，我们从理论上评估了各分支的时间与空间复杂度，并在GPU内存占用、吞吐量和收敛性方面进行了实证对比。进一步地，我们分析了可扩展GNNs各分支的优缺点，并提出了一种名为EnGCN的新型集成训练方法，以解决现有问题。我们的代码公开在 https://github.com/VITA-Group/Large_Scale_GCN_Benchmarking。