Message-passing graph neural networks (MPNNs) have emerged as a powerful paradigm for graph-based machine learning. Despite their effectiveness, MPNNs face challenges such as under-reaching and over-squashing, where limited receptive fields and structural bottlenecks hinder information flow in the graph. While graph transformers hold promise in addressing these issues, their scalability is limited due to quadratic complexity regarding the number of nodes, rendering them impractical for larger graphs. Here, we propose \emph{implicitly rewired message-passing neural networks} (IPR-MPNNs), a novel approach that integrates \emph{implicit} probabilistic graph rewiring into MPNNs. By introducing a small number of virtual nodes, i.e., adding additional nodes to a given graph and connecting them to existing nodes, in a differentiable, end-to-end manner, IPR-MPNNs enable long-distance message propagation, circumventing quadratic complexity. Theoretically, we demonstrate that IPR-MPNNs surpass the expressiveness of traditional MPNNs. Empirically, we validate our approach by showcasing its ability to mitigate under-reaching and over-squashing effects, achieving state-of-the-art performance across multiple graph datasets. Notably, IPR-MPNNs outperform graph transformers while maintaining significantly faster computational efficiency.

翻译：消息传递图神经网络（MPNNs）已成为图机器学习的有力范式。尽管其效果显著，MPNNs仍面临诸如"欠达"与"过压缩"等挑战——有限的感受野与结构瓶颈阻碍了图中信息流动。虽然图Transformer有望解决这些问题，但其可扩展性受限于节点数量的二次复杂度，难以应用于大规模图。本文提出一种新颖方法——隐式重布线消息传递神经网络（IPR-MPNNs），该方法将隐式概率图重布线机制集成到MPNNs中。通过以可微分、端到端的方式引入少量虚拟节点（即在给定图中添加额外节点并将其与现有节点连接），IPR-MPNNs实现了长距离消息传播，同时规避了二次复杂度。理论上，我们证明IPR-MPNNs的表达能力超越了传统MPNNs。实证方面，我们通过展示该方法缓解欠达与过压缩效应的能力，在多个图数据集上取得了最先进的性能，验证了其有效性。值得注意的是，IPR-MPNNs在保持显著更快计算效率的同时，性能超越了图Transformer。