Graph Neural Networks (GNNs) are the state-of-the-art model for machine learning on graph-structured data. The most popular class of GNNs operate by exchanging information between adjacent nodes, and are known as Message Passing Neural Networks (MPNNs). Given their widespread use, understanding the expressive power of MPNNs is a key question. However, existing results typically consider settings with uninformative node features. In this paper, we provide a rigorous analysis to determine which function classes of node features can be learned by an MPNN of a given capacity. We do so by measuring the level of pairwise interactions between nodes that MPNNs allow for. This measure provides a novel quantitative characterization of the so-called over-squashing effect, which is observed to occur when a large volume of messages is aggregated into fixed-size vectors. Using our measure, we prove that, to guarantee sufficient communication between pairs of nodes, the capacity of the MPNN must be large enough, depending on properties of the input graph structure, such as commute times. For many relevant scenarios, our analysis results in impossibility statements in practice, showing that over-squashing hinders the expressive power of MPNNs. We validate our theoretical findings through extensive controlled experiments and ablation studies.

翻译：图神经网络（GNN）是针对图结构数据进行机器学习的最先进模型。最流行的GNN类别通过相邻节点间的信息交换运作，被称为消息传递神经网络（MPNN）。鉴于其广泛应用，理解MPNN的表达能力是一个关键问题。然而，现有结果通常考虑节点特征信息有限的设置。本文提供了一种严格的分析方法，用以确定何种节点特征函数类能够被给定容量的MPNN学习。我们通过测量MPNN允许的节点间成对交互水平来实现这一目标。这一测量方法为所谓的过度挤压效应提供了新的定量表征——当大量信息被聚合到固定尺寸的向量中时，观察到该效应发生。利用我们的测量方法，我们证明：为保证节点间的充分通信，MPNN的容量必须足够大，其取决于输入图结构属性（如游走时间）。对于许多相关场景，我们的分析得出了实际中不可能的结论，表明过度挤压阻碍了MPNN的表达能力。我们通过大量受控实验和消融研究验证了理论发现。