Graph transformers have emerged as a promising architecture for a variety of graph learning and representation tasks. Despite their successes, though, it remains challenging to scale graph transformers to large graphs while maintaining accuracy competitive with message-passing networks. In this paper, we introduce Exphormer, a framework for building powerful and scalable graph transformers. Exphormer consists of a sparse attention mechanism based on two mechanisms: virtual global nodes and expander graphs, whose mathematical characteristics, such as spectral expansion, pseduorandomness, and sparsity, yield graph transformers with complexity only linear in the size of the graph, while allowing us to prove desirable theoretical properties of the resulting transformer models. We show that incorporating Exphormer into the recently-proposed GraphGPS framework produces models with competitive empirical results on a wide variety of graph datasets, including state-of-the-art results on three datasets. We also show that Exphormer can scale to datasets on larger graphs than shown in previous graph transformer architectures. Code can be found at \url{https://github.com/hamed1375/Exphormer}.

翻译：图Transformer已成为多种图学习与表示任务中颇具前景的架构。然而，尽管取得了成功，如何在保持与消息传递网络相竞争的准确性的同时，将图Transformer扩展至大规模图仍然具有挑战性。本文提出了Exphormer——一个构建强大且可扩展图Transformer的框架。Exphormer包含一种基于两大机制的稀疏注意力机制：虚拟全局节点和扩展图，其数学特性（如谱扩展、伪随机性和稀疏性）使得图Transformer的复杂度仅与图大小呈线性关系，同时使我们能够证明所得Transformer模型具有理想的理论性质。我们表明，将Exphormer集成到近期提出的GraphGPS框架中，可在多种图数据集上产生具有竞争力的实证结果，包括在三个数据集上取得最优结果。我们还证明，Exphormer能够扩展至比以往图Transformer架构所展示的更大规模的图数据集。代码见\url{https://github.com/hamed1375/Exphormer}。