The increasing size of input graphs for graph neural networks (GNNs) highlights the demand for using multi-GPU platforms. However, existing multi-GPU GNN systems optimize the computation and communication individually based on the conventional practice of scaling dense DNNs. For irregularly sparse and fine-grained GNN workloads, such solutions miss the opportunity to jointly schedule/optimize the computation and communication operations for high-performance delivery. To this end, we propose MGG, a novel system design to accelerate full-graph GNNs on multi-GPU platforms. The core of MGG is its novel dynamic software pipeline to facilitate fine-grained computation-communication overlapping within a GPU kernel. Specifically, MGG introduces GNN-tailored pipeline construction and GPU-aware pipeline mapping to facilitate workload balancing and operation overlapping. MGG also incorporates an intelligent runtime design with analytical modeling and optimization heuristics to dynamically improve the execution performance. Extensive evaluation reveals that MGG outperforms state-of-the-art full-graph GNN systems across various settings: on average 4.41X, 4.81X, and 10.83X faster than DGL, MGG-UVM, and ROC, respectively.

翻译：图神经网络（GNN）输入图规模的不断增大凸显了对多GPU平台的需求。然而，现有的多GPU GNN系统基于传统密集型深度神经网络的扩展实践，将计算与通信分别进行优化。对于不规则稀疏且细粒度的GNN工作负载，此类方案无法协同调度/优化计算与通信操作以实现高性能交付。为此，我们提出MGG——一种用于在多GPU平台上加速全图GNN的新型系统设计方案。MGG的核心是其创新的动态软件流水线，可在GPU核内实现细粒度的计算-通信重叠。具体而言，MGG引入了面向GNN的流水线构建策略与面向GPU的流水线映射机制，以促进工作负载均衡与操作重叠。此外，MGG还集成了包含分析建模与优化启发式方法的智能运行时设计，可动态提升执行性能。大量评估表明，MGG在不同配置下均优于当前最先进的全图GNN系统：相比DGL、MGG-UVM和ROC分别平均加速4.41倍、4.81倍和10.83倍。