Graph Transformers (GTs) have significantly advanced the field of graph representation learning by overcoming the limitations of message-passing graph neural networks (GNNs) and demonstrating promising performance and expressive power. However, the quadratic complexity of self-attention mechanism in GTs has limited their scalability, and previous approaches to address this issue often suffer from expressiveness degradation or lack of versatility. To address this issue, we propose AnchorGT, a novel attention architecture for GTs with global receptive field and almost linear complexity, which serves as a flexible building block to improve the scalability of a wide range of GT models. Inspired by anchor-based GNNs, we employ structurally important $k$-dominating node set as anchors and design an attention mechanism that focuses on the relationship between individual nodes and anchors, while retaining the global receptive field for all nodes. With its intuitive design, AnchorGT can easily replace the attention module in various GT models with different network architectures and structural encodings, resulting in reduced computational overhead without sacrificing performance. In addition, we theoretically prove that AnchorGT attention can be strictly more expressive than Weisfeiler-Lehman test, showing its superiority in representing graph structures. Our experiments on three state-of-the-art GT models demonstrate that their AnchorGT variants can achieve better results while being faster and significantly more memory efficient.

翻译：摘要：图 Transformer（GTs）通过克服消息传递图神经网络（GNNs）的局限性，显著推进了图表示学习领域的发展，展现出卓越的性能和表达能力。然而，GTs 中自注意力机制的二次复杂度限制了其可扩展性，而先前解决此问题的方法常伴随着表达能力下降或缺乏通用性。为此，我们提出 AnchorGT，一种新型 GT 注意力架构，具备全局感受野和近似线性复杂度，可作为灵活构建块提升多种 GT 模型的可扩展性。受基于锚点的 GNNs 启发，我们采用结构重要的 $k$-支配节点集作为锚点，并设计了一种关注单个节点与锚点之间关系的注意力机制，同时为所有节点保留全局感受野。凭借其直观设计，AnchorGT 可轻松替换不同网络架构和结构编码的 GT 模型中的注意力模块，从而在降低计算开销的同时不牺牲性能。此外，我们从理论上证明，AnchorGT 注意力在表达能力上严格强于 Weisfeiler-Lehman 测试，展现了其在图结构表示中的优越性。我们在三种最先进的 GT 模型上的实验表明，它们的 AnchorGT 变体能在实现更优结果的同时，速度更快且内存效率显著提升。