Graph neural networks are widely used tools for graph prediction tasks. Motivated by their empirical performance, prior works have developed generalization bounds for graph neural networks, which scale with graph structures in terms of the maximum degree. In this paper, we present generalization bounds that instead scale with the largest singular value of the graph neural network's feature diffusion matrix. These bounds are numerically much smaller than prior bounds for real-world graphs. We also construct a lower bound of the generalization gap that matches our upper bound asymptotically. To achieve these results, we analyze a unified model that includes prior works' settings (i.e., convolutional and message-passing networks) and new settings (i.e., graph isomorphism networks). Our key idea is to measure the stability of graph neural networks against noise perturbations using Hessians. Empirically, we find that Hessian-based measurements correlate with the observed generalization gaps of graph neural networks accurately; Optimizing noise stability properties for fine-tuning pretrained graph neural networks also improves test performance on several graph-level classification tasks.

翻译：图神经网络是广泛用于图预测任务的工具。受其经验性能驱动，先前工作针对图神经网络建立了泛化界，这些界以最大度数为尺度随图结构变化。本文提出了以图神经网络特征扩散矩阵的最大奇异值为尺度的泛化界。对于真实世界图，该数值远小于先前研究中的界。我们还构建了与上界渐近匹配的泛化间隙下界。为实现这些结果，我们分析了一个统一模型，该模型涵盖先前工作设置（即卷积网络和消息传递网络）及新设置（即图同构网络）。我们的核心思想是利用Hessian矩阵衡量图神经网络对噪声扰动的稳定性。实验发现，基于Hessian的测度能准确关联图神经网络观测到的泛化间隙；通过优化噪声稳定性属性微调预训练图神经网络，也能提升多个图级分类任务的测试性能。