Graph neural networks are widely used tools for graph prediction tasks. Motivated by their empirical performance, prior works have developed generalization bounds for graph neural networks, which scale with graph structures in terms of the maximum degree. In this paper, we present generalization bounds that instead scale with the largest singular value of the graph neural network's feature diffusion matrix. These bounds are numerically much smaller than prior bounds for real-world graphs. We also construct a lower bound of the generalization gap that matches our upper bound asymptotically. To achieve these results, we analyze a unified model that includes prior works' settings (i.e., convolutional and message-passing networks) and new settings (i.e., graph isomorphism networks). Our key idea is to measure the stability of graph neural networks against noise perturbations using Hessians. Empirically, we find that Hessian-based measurements correlate with the observed generalization gaps of graph neural networks accurately. Optimizing noise stability properties for fine-tuning pretrained graph neural networks also improves test performance on several graph-level classification tasks.

翻译：图神经网络是广泛应用于图预测任务的工具。受其经验性能的启发，先前的工作已为图神经网络建立了泛化界，这些界根据图结构中的最大度进行缩放。在本文中，我们提出了另一种泛化界，它改为根据图神经网络特征扩散矩阵的最大奇异值进行缩放。对于真实世界的图，这些界在数值上远小于先前的界。我们还构建了一个与上界渐近匹配的泛化差距下界。为获得这些结果，我们分析了一个统一模型，该模型涵盖了先前工作的设置（即卷积网络和消息传递网络）以及新的设置（即图同构网络）。我们的核心思想是利用海森矩阵衡量图神经网络对噪声扰动的稳定性。实验发现，基于海森矩阵的度量能准确关联图神经网络观察到的泛化差距。优化噪声稳定性属性以微调预训练的图神经网络，还能在多个图级分类任务上提升测试性能。