Graph neural networks (GNNs) achieve remarkable performance in graph machine learning tasks but can be hard to train on large-graph data, where their learning dynamics are not well understood. We investigate the training dynamics of large-graph GNNs using graph neural tangent kernels (GNTKs) and graphons. In the limit of large width, optimization of an overparametrized NN is equivalent to kernel regression on the NTK. Here, we investigate how the GNTK evolves as another independent dimension is varied: the graph size. We use graphons to define limit objects -- graphon NNs for GNNs, and graphon NTKs for GNTKs -- , and prove that, on a sequence of graphs, the GNTKs converge to the graphon NTK. We further prove that the spectrum of the GNTK, which is related to the directions of fastest learning which becomes relevant during early stopping, converges to the spectrum of the graphon NTK. This implies that in the large-graph limit, the GNTK fitted on a graph of moderate size can be used to solve the same task on the large graph, and to infer the learning dynamics of the large-graph GNN. These results are verified empirically on node regression and classification tasks.

翻译：图神经网络（GNN）在图机器学习任务中取得了显著性能，但在大规模图数据上训练困难，其学习动态尚未被充分理解。我们利用图神经正切核（GNTK）和图极限（graphon）研究大规模图GNN的训练动态。在宽度趋于无穷的极限下，过参数化神经网络的优化等价于神经正切核（NTK）上的核回归。本文探究当图规模作为另一独立维度变化时GNTK的演化规律。我们使用图极限定义极限对象——针对GNN的图极限神经网络（graphon NN）与针对GNTK的图极限正切核（graphon NTK），并证明在图序列上GNTK收敛至图极限NTK。进一步证明，与早停阶段最快学习方向相关的GNTK谱收敛至图极限NTK谱。这意味着在大图极限下，基于中等规模图拟合的GNTK可适用于大规模图的同类任务，并推断大图GNN的学习动态。我们在节点回归与分类任务上通过实验验证了上述结论。