Graph neural networks (GNNs), as the de-facto model class for representation learning on graphs, are built upon the multi-layer perceptrons (MLP) architecture with additional message passing layers to allow features to flow across nodes. While conventional wisdom commonly attributes the success of GNNs to their advanced expressivity, we conjecture that this is not the main cause of GNNs' superiority in node-level prediction tasks. This paper pinpoints the major source of GNNs' performance gain to their intrinsic generalization capability, by introducing an intermediate model class dubbed as P(ropagational)MLP, which is identical to standard MLP in training, but then adopts GNN's architecture in testing. Intriguingly, we observe that PMLPs consistently perform on par with (or even exceed) their GNN counterparts, while being much more efficient in training. This finding sheds new insights into understanding the learning behavior of GNNs, and can be used as an analytic tool for dissecting various GNN-related research problems. As an initial step to analyze the inherent generalizability of GNNs, we show the essential difference between MLP and PMLP at infinite-width limit lies in the NTK feature map in the post-training stage. Moreover, by examining their extrapolation behavior, we find that though many GNNs and their PMLP counterparts cannot extrapolate non-linear functions for extremely out-of-distribution samples, they have greater potential to generalize to testing samples near the training data range as natural advantages of GNN architectures.

翻译：图神经网络（GNN）作为图表示学习的事实标准模型类，基于多层感知器（MLP）架构构建，并额外引入消息传递层以实现节点间的特征流动。尽管传统观点通常将GNN的成功归因于其强大的表达能力，但我们推测这并非GNN在节点级预测任务中优越性的主要原因。本文通过引入名为P(ropagational)MLP的中间模型类，明确指出GNN性能提升的主要来源是其内在的泛化能力——该模型在训练时与标准MLP完全相同，但在测试阶段采用GNN架构。引人注目的是，我们观察到PMLP在保持与GNN相当（甚至超越其）性能的同时，训练效率大幅提升。这一发现为理解GNN的学习行为提供了新洞见，并可作为剖析各类GNN相关研究问题的分析工具。作为分析GNN内在泛化能力的初步探索，我们揭示了无限宽极限下MLP与PMLP的本质差异在于训练后阶段的NTK特征映射。此外，通过考察其外推行为，我们发现：尽管多数GNN及其对应的PMLP无法对极端分布外样本进行非线性函数外推，但作为GNN架构的自然优势，它们具有更强的泛化潜力，能够对训练数据范围内的测试样本实现有效泛化。