Message Passing Neural Networks (MPNNs) are instances of Graph Neural Networks that leverage the graph to send messages over the edges. This inductive bias leads to a phenomenon known as over-squashing, where a node feature is insensitive to information contained at distant nodes. Despite recent methods introduced to mitigate this issue, an understanding of the causes for over-squashing and of possible solutions are lacking. In this theoretical work, we prove that: (i) Neural network width can mitigate over-squashing, but at the cost of making the whole network more sensitive; (ii) Conversely, depth cannot help mitigate over-squashing: increasing the number of layers leads to over-squashing being dominated by vanishing gradients; (iii) The graph topology plays the greatest role, since over-squashing occurs between nodes at high commute (access) time. Our analysis provides a unified framework to study different recent methods introduced to cope with over-squashing and serves as a justification for a class of methods that fall under graph rewiring.

翻译：消息传递神经网络（MPNNs）是图神经网络的一种实例，通过利用图结构在边上传递消息。这种归纳偏置会导致一种称为“过压缩”的现象，即节点特征对远端节点所含信息不敏感。尽管近期已提出缓解该问题的方法，但对其成因及可行解决方案的理解仍显不足。在这项理论工作中，我们证明了：(i) 神经网络宽度可以缓解过压缩，但代价是使整个网络更敏感；(ii) 反之，深度无法帮助缓解过压缩：增加层数会导致过压缩被梯度消失主导；(iii) 图拓扑结构起着最重要的作用，因为过压缩发生在高通勤（访问）时间的节点之间。我们的分析为研究近期提出的多种过压缩应对方法提供了统一框架，并论证了图重连类方法的合理性。