The graph Transformer emerges as a new architecture and has shown superior performance on various graph mining tasks. In this work, we observe that existing graph Transformers treat nodes as independent tokens and construct a single long sequence composed of all node tokens so as to train the Transformer model, causing it hard to scale to large graphs due to the quadratic complexity on the number of nodes for the self-attention computation. To this end, we propose a Neighborhood Aggregation Graph Transformer (NAGphormer) that treats each node as a sequence containing a series of tokens constructed by our proposed Hop2Token module. For each node, Hop2Token aggregates the neighborhood features from different hops into different representations and thereby produces a sequence of token vectors as one input. In this way, NAGphormer could be trained in a mini-batch manner and thus could scale to large graphs. Moreover, we mathematically show that as compared to a category of advanced Graph Neural Networks (GNNs), the decoupled Graph Convolutional Network, NAGphormer could learn more informative node representations from the multi-hop neighborhoods. Extensive experiments on benchmark datasets from small to large are conducted to demonstrate that NAGphormer consistently outperforms existing graph Transformers and mainstream GNNs. Code is available at https://github.com/JHL-HUST/NAGphormer.

翻译：图Transformer作为一种新架构，已在各类图挖掘任务中展现出卓越性能。本文观察到，现有图Transformer将节点视为独立标记，并构建包含所有节点标记的单一长序列来训练Transformer模型，但由于自注意力计算对节点数量具有二次复杂度，该方法难以扩展至大规模图。为此，我们提出邻域聚合图Transformer（NAGphormer），该模型将每个节点视为由所提出的Hop2Token模块构建的一系列标记组成的序列。对于每个节点，Hop2Token将来自不同跳数的邻域特征聚合为不同的表示，从而生成一个标记向量序列作为输入。通过这种方式，NAGphormer可以采用小批量方式训练，因而能够扩展至大规模图。此外，我们从数学上证明，与一类先进的图神经网络（GNN）——解耦图卷积网络相比，NAGphormer能从多跳邻域中学习到更具信息量的节点表示。我们在从中小规模到大规模基准数据集上进行了广泛实验，结果表明NAGphormer始终优于现有的图Transformer和主流GNN。代码已开源至https://github.com/JHL-HUST/NAGphormer。