Message Passing Neural Networks (MPNNs) are instances of Graph Neural Networks that leverage the graph to send messages over the edges. This inductive bias leads to a phenomenon known as over-squashing, where a node feature is insensitive to information contained at distant nodes. Despite recent methods introduced to mitigate this issue, an understanding of the causes for over-squashing and of possible solutions are lacking. In this theoretical work, we prove that: (i) Neural network width can mitigate over-squashing, but at the cost of making the whole network more sensitive; (ii) Conversely, depth cannot help mitigate over-squashing: increasing the number of layers leads to over-squashing being dominated by vanishing gradients; (iii) The graph topology plays the greatest role, since over-squashing occurs between nodes at high commute (access) time. Our analysis provides a unified framework to study different recent methods introduced to cope with over-squashing and serves as a justification for a class of methods that fall under `graph rewiring'.

翻译：消息传递神经网络（MPNNs）是图神经网络的一种实例，其利用图结构沿边传输消息。这种归纳偏置会导致一种称为“过挤压”的现象，即节点特征对远端节点所包含的信息不敏感。尽管近期已有多种方法被提出以缓解该问题，但对过挤压成因及潜在解决方案的理解仍存在不足。在这项理论工作中，我们证明：（i）神经网络宽度可以缓解过挤压，但代价是使整个网络更敏感；（ii）相反，深度无法帮助缓解过挤压：增加层数会导致过挤压被梯度消失问题所主导；（iii）图拓扑结构起决定性作用，因为过挤压出现在具有高通勤（访问）时间的节点之间。我们的分析为研究近期应对过挤压的不同方法提供了统一框架，并为属于“图重连”类别的方法提供了理论依据。