Graph Neural Networks (GNNs) have shown success in many real-world applications that involve graph-structured data. Most of the existing single-node GNN training systems are capable of training medium-scale graphs with tens of millions of edges; however, scaling them to large-scale graphs with billions of edges remains challenging. In addition, it is challenging to map GNN training algorithms onto a computation node as state-of-the-art machines feature heterogeneous architecture consisting of multiple processors and a variety of accelerators. We propose HyScale-GNN, a novel system to train GNN models on a single-node heterogeneous architecture. HyScale- GNN performs hybrid training which utilizes both the processors and the accelerators to train a model collaboratively. Our system design overcomes the memory size limitation of existing works and is optimized for training GNNs on large-scale graphs. We propose a two-stage data pre-fetching scheme to reduce the communication overhead during GNN training. To improve task mapping efficiency, we propose a dynamic resource management mechanism, which adjusts the workload assignment and resource allocation during runtime. We evaluate HyScale-GNN on a CPU-GPU and a CPU-FPGA heterogeneous architecture. Using several large-scale datasets and two widely-used GNN models, we compare the performance of our design with a multi-GPU baseline implemented in PyTorch-Geometric. The CPU-GPU design and the CPU-FPGA design achieve up to 2.08x speedup and 12.6x speedup, respectively. Compared with the state-of-the-art large-scale multi-node GNN training systems such as P3 and DistDGL, our CPU-FPGA design achieves up to 5.27x speedup using a single node.

翻译：图神经网络（GNN）在处理图结构数据的诸多实际应用中已展现出有效性。现有单节点GNN训练系统虽能处理千万级边规模的中等规模图，但将其扩展到十亿级边的大规模图仍面临挑战。此外，由于当前最先进的计算节点通常采用包含多处理器与多种加速器的异构架构，如何将GNN训练算法高效映射至计算节点仍存在困难。我们提出HyScale-GNN这一新型系统，用于在单节点异构架构上训练GNN模型。HyScale-GNN采用混合训练方式，协同利用处理器与加速器共同完成模型训练。该系统设计突破了现有方法的内存容量限制，并针对大规模图的GNN训练进行了优化。我们提出两阶段数据预取方案以降低GNN训练中的通信开销。为提升任务映射效率，我们设计动态资源管理机制，在运行时调整工作负载分配与资源配置。我们在CPU-GPU与CPU-FPGA两种异构架构上评估了HyScale-GNN。采用多个大规模数据集与两种广泛使用的GNN模型，我们将系统性能与基于PyTorch-Geometric实现的多GPU基准方案进行对比。CPU-GPU方案与CPU-FPGA方案分别实现了最高2.08倍和12.6倍的加速比。与P3、DistDGL等现有最先进的大规模多节点GNN训练系统相比，我们的CPU-FPGA方案在单节点上取得了最高5.27倍的加速比。