Graph Neural Networks (GNNs) are the state-of-the-art model for machine learning on graph-structured data. The most popular class of GNNs operate by exchanging information between adjacent nodes, and are known as Message Passing Neural Networks (MPNNs). Given their widespread use, understanding the expressive power of MPNNs is a key question. However, existing results typically consider settings with uninformative node features. In this paper, we provide a rigorous analysis to determine which function classes of node features can be learned by an MPNN of a given capacity. We do so by measuring the level of pairwise interactions between nodes that MPNNs allow for. This measure provides a novel quantitative characterization of the so-called over-squashing effect, which is observed to occur when a large volume of messages is aggregated into fixed-size vectors. Using our measure, we prove that, to guarantee sufficient communication between pairs of nodes, the capacity of the MPNN must be large enough, depending on properties of the input graph structure, such as commute times. For many relevant scenarios, our analysis results in impossibility statements in practice, showing that over-squashing hinders the expressive power of MPNNs. We validate our theoretical findings through extensive controlled experiments and ablation studies.

翻译：图神经网络（GNN）是处理图结构数据的机器学习最先进模型。最常用的GNN通过相邻节点间交换信息来运作，这类模型被称为消息传递神经网络（MPNN）。鉴于其广泛应用，理解MPNN的表达能力是一个关键问题。然而，现有结果通常考虑节点特征无信息量的设置。本文中，我们提供了严格的分析，以确定给定容量的MPNN能学习哪些节点特征的函数类别。我们通过衡量MPNN允许的节点间成对交互水平来完成这一分析。这种度量提供了对所谓"过压缩"效应的新颖定量刻画，该效应在大规模消息被聚合成固定大小向量时观察到。利用我们的度量，我们证明：为保证节点对之间充分通信，MPNN的容量必须足够大，这取决于输入图结构的性质（如通勤时间）。对于许多相关场景，我们的分析在实际中得出不可能性结论，表明过压缩阻碍了MPNN的表达能力。我们通过大量受控实验和消融研究验证了理论发现。