Graph neural networks (GNNs) achieve remarkable performance in graph machine learning tasks but can be hard to train on large-graph data, where their learning dynamics are not well understood. We investigate the training dynamics of large-graph GNNs using graph neural tangent kernels (GNTKs) and graphons. In the limit of large width, optimization of an overparametrized NN is equivalent to kernel regression on the NTK. Here, we investigate how the GNTK evolves as another independent dimension is varied: the graph size. We use graphons to define limit objects -- graphon NNs for GNNs, and graphon NTKs for GNTKs, and prove that, on a sequence of growing graphs, the GNTKs converge to the graphon NTK. We further prove that the eigenspaces of the GNTK, which are related to the problem learning directions and associated learning speeds, converge to the spectrum of the GNTK. This implies that in the large-graph limit, the GNTK fitted on a graph of moderate size can be used to solve the same task on the large-graph and infer the learning dynamics of the large-graph GNN. These results are verified empirically on node regression and node classification tasks.

翻译：图神经网络（GNN）在图机器学习任务中表现出色，但在大图数据上可能难以训练，其学习动态尚未被充分理解。本文利用图神经正切核（GNTK）和图核（graphons）研究大图GNN的训练动态。在宽度趋于无穷的极限下，过参数化神经网络的优化等价于NTK上的核回归。本文探讨当另一独立维度——图规模变化时，GNTK的演化规律。我们使用图核定义极限对象：GNN的图核神经网络（graphon NN）以及GNTK的图核正切核（graphon NTK），并证明在递增图序列上，GNTK收敛至图核NTK。进一步证明，GNTK的特征空间（与问题学习方向及相应学习速度相关）收敛至GNTK的谱。这表明，在大图极限下，基于中等规模图拟合的GNTK可用于求解相同任务的大图问题，并推断大图GNN的学习动态。这些结果在节点回归和节点分类任务上得到了实验验证。