Message Passing Neural Networks (MPNNs) are instances of Graph Neural Networks that leverage the graph to send messages over the edges. This inductive bias leads to a phenomenon known as over-squashing, where a node feature is insensitive to information contained at distant nodes. Despite recent methods introduced to mitigate this issue, an understanding of the causes for over-squashing and of possible solutions are lacking. In this theoretical work, we prove that: (i) Neural network width can mitigate over-squashing, but at the cost of making the whole network more sensitive; (ii) Conversely, depth cannot help mitigate over-squashing: increasing the number of layers leads to over-squashing being dominated by vanishing gradients; (iii) The graph topology plays the greatest role, since over-squashing occurs between nodes at high commute (access) time. Our analysis provides a unified framework to study different recent methods introduced to cope with over-squashing and serves as a justification for a class of methods that fall under `graph rewiring'.

翻译：消息传递神经网络（MPNNs）是图神经网络的一种实例，它利用图结构沿边传递消息。这种归纳偏置导致了一种称为“过度压缩”的现象，即节点特征对远端节点所包含的信息不敏感。尽管近期已有方法被提出以缓解该问题，但关于过度压缩的成因及潜在解决方案的理解仍较为欠缺。在本理论工作中，我们证明：（i）神经网络宽度可以缓解过度压缩，但代价是使得整个网络更加敏感；（ii）相反地，深度无法帮助缓解过度压缩：增加层数会导致过度压缩主要由梯度消失主导；（iii）图拓扑结构发挥最重要作用，因为过度压缩出现在具有高通勤（访问）时间的节点之间。我们的分析为研究近期引入的多种应对过度压缩的方法提供了统一框架，并为一类属于“图重连”的方法提供了理论依据。