Graph Neural Networks (GNNs) are the state-of-the-art model for machine learning on graph-structured data. The most popular class of GNNs operate by exchanging information between adjacent nodes, and are known as Message Passing Neural Networks (MPNNs). Given their widespread use, understanding the expressive power of MPNNs is a key question. However, existing results typically consider settings with uninformative node features. In this paper, we provide a rigorous analysis to determine which function classes of node features can be learned by an MPNN of a given capacity. We do so by measuring the level of pairwise interactions between nodes that MPNNs allow for. This measure provides a novel quantitative characterization of the so-called over-squashing effect, which is observed to occur when a large volume of messages is aggregated into fixed-size vectors. Using our measure, we prove that, to guarantee sufficient communication between pairs of nodes, the capacity of the MPNN must be large enough, depending on properties of the input graph structure, such as commute times. For many relevant scenarios, our analysis results in impossibility statements in practice, showing that over-squashing hinders the expressive power of MPNNs. We validate our theoretical findings through extensive controlled experiments and ablation studies.

翻译：图神经网络（GNN）是处理图结构数据机器学习任务的最先进模型。最流行的GNN类别通过相邻节点间的信息交换来运行，被称为消息传递神经网络（MPNN）。鉴于其广泛应用，理解MPNN的表达能力成为关键问题。然而，现有结果通常考虑节点特征无信息量的设定。本文提供了严格的分析，以确定给定容量的MPNN能够学习哪些节点特征函数类别。我们通过测量MPNN允许的节点间成对交互水平来实现这一目标。该测量方法为所谓”过度压缩效应”提供了新颖的定量刻画——当大量消息被聚合为固定尺寸向量时，观测到这种效应。利用该测量方法，我们证明：为保证节点对间的充分通信，MPNN的容量必须足够大（取决于输入图结构的属性，如游走时间）。针对许多相关场景，我们的分析在实际中得出不可能性结论，表明过度压缩阻碍了MPNN的表达能力。通过广泛的控制实验和消融研究，我们验证了理论发现。