Neural tangent kernels (NTKs) provide a theoretical regime to analyze the learning and generalization behavior of over-parametrized neural networks. For a supervised learning task, the association between the eigenvectors of the NTK kernel and given data (a concept referred to as alignment in this paper) can govern the rate of convergence of gradient descent, as well as generalization to unseen data. Building upon this concept, we investigate NTKs and alignment in the context of graph neural networks (GNNs), where our analysis reveals that optimizing alignment translates to optimizing the graph representation or the graph shift operator in a GNN. Our results further establish the theoretical guarantees on the optimality of the alignment for a two-layer GNN and these guarantees are characterized by the graph shift operator being a function of the cross-covariance between the input and the output data. The theoretical insights drawn from the analysis of NTKs are validated by our experiments focused on a multi-variate time series prediction task for a publicly available dataset. Specifically, they demonstrate that GNNs with cross-covariance as the graph shift operator indeed outperform those that operate on the covariance matrix from only the input data.

翻译：神经正切核（NTKs）为分析过参数化神经网络的学习与泛化行为提供了理论框架。在监督学习任务中，NTK核的特征向量与给定数据之间的关联（本文中称为对齐概念）可主导梯度下降的收敛速度以及对未知数据的泛化能力。基于这一概念，我们研究了图神经网络（GNN）背景下的NTK与对齐现象，分析表明优化对齐等价于优化GNN中的图表示或图移位算子。研究结果进一步建立了双层GNN对齐最优性的理论保证，这些保证由作为输入数据与输出数据间交叉协方差函数的图移位算子来表征。基于NTK分析得出的理论洞见通过面向公开数据集的多变量时间序列预测实验得到验证。具体而言，实验表明采用交叉协方差作为图移位算子的GNN确实优于仅基于输入数据协方差矩阵运行的GNN模型。