Graph neural networks are widely used tools for graph prediction tasks. Motivated by their empirical performance, prior works have developed generalization bounds for graph neural networks, which scale with graph structures in terms of the maximum degree. In this paper, we present generalization bounds that instead scale with the largest singular value of the graph neural network's feature diffusion matrix. These bounds are numerically much smaller than prior bounds for real-world graphs. We also construct a lower bound of the generalization gap that matches our upper bound asymptotically. To achieve these results, we analyze a unified model that includes prior works' settings (i.e., convolutional and message-passing networks) and new settings (i.e., graph isomorphism networks). Our key idea is to measure the stability of graph neural networks against noise perturbations using Hessians. Empirically, we find that Hessian-based measurements correlate with the observed generalization gaps of graph neural networks accurately. Optimizing noise stability properties for fine-tuning pretrained graph neural networks also improves test performance on several graph-level classification tasks.

翻译：图神经网络是广泛应用于图预测任务的工具。受其经验性能的启发，先前工作已为图神经网络建立了泛化界，这些界以最大度数为指标与图结构成比例缩放。本文提出了与图神经网络特征扩散矩阵的最大奇异值成比例缩放的泛化界。对于真实世界图数据，这些数值远小于先前的泛化界。我们还构造了与上界渐近匹配的泛化差距下界。为获得这些结果，我们分析了包含先前工作设定（即卷积网络和消息传递网络）与新设定（即图同构网络）的统一模型。核心思想是利用Hessian矩阵度量图神经网络对噪声扰动的稳定性。实验表明，基于Hessian的测量值与观测到的图神经网络泛化差距具有准确相关性。在微调预训练图神经网络时优化噪声稳定性属性，还能提升多个图级分类任务的测试性能。