Graph neural networks (GNNs) are machine learning models specialized for graph data and widely used in many applications. To train GNNs on large graphs that exceed CPU memory, several systems store data on disk and conduct out-of-core processing. However, these systems suffer from either read amplification when reading node features that are usually smaller than a disk page or degraded model accuracy by treating the graph as disconnected partitions. To close this gap, we build a system called DiskGNN, which achieves high I/O efficiency and thus fast training without hurting model accuracy. The key technique used by DiskGNN is offline sampling, which helps decouple graph sampling from model computation. In particular, by conducting graph sampling beforehand, DiskGNN acquires the node features that will be accessed by model computation, and such information is utilized to pack the target node features contiguously on disk to avoid read amplification. Besides, \name{} also adopts designs including four-level feature store to fully utilize the memory hierarchy to cache node features and reduce disk access, batched packing to accelerate the feature packing process, and pipelined training to overlap disk access with other operations. We compare DiskGNN with Ginex and MariusGNN, which are state-of-the-art systems for out-of-core GNN training. The results show that DiskGNN can speed up the baselines by over 8x while matching their best model accuracy.

翻译：图神经网络(GNN)是专门针对图数据的机器学习模型，广泛应用于众多领域。为了在超出CPU内存的大规模图上训练GNN，许多系统将数据存储在磁盘上并进行内存外处理。然而，这些系统面临要么在读取节点特征（通常小于磁盘页面）时出现读取放大问题，要么因将图视为不连通的分区导致模型精度下降。为弥补这一差距，我们构建了名为DiskGNN的系统，该系统在不损害模型精度的前提下实现了高I/O效率，从而加速训练。DiskGNN采用的关键技术是离线采样，它有助于将图采样与模型计算解耦。具体而言，通过预先进行图采样，DiskGNN能够获取模型计算将要访问的节点特征，并利用该信息将目标节点特征在磁盘上连续打包存储，以避免读取放大。此外，该系统还采用四级特征存储等设计，以充分利用内存层次结构缓存节点特征并减少磁盘访问；采用批量打包加速特征打包过程；以及采用流水线训练使磁盘访问与其他操作重叠。我们对比了DiskGNN与当前最先进的内存外GNN训练系统Ginex和MariusGNN。结果表明，DiskGNN在匹配基线方法最优模型精度的前提下，可将训练速度提升超过8倍。