While tokenized graph Transformers have demonstrated strong performance in node classification tasks, their reliance on a limited subset of nodes with high similarity scores for constructing token sequences overlooks valuable information from other nodes, hindering their ability to fully harness graph information for learning optimal node representations. To address this limitation, we propose a novel graph Transformer called GCFormer. Unlike previous approaches, GCFormer develops a hybrid token generator to create two types of token sequences, positive and negative, to capture diverse graph information. And a tailored Transformer-based backbone is adopted to learn meaningful node representations from these generated token sequences. Additionally, GCFormer introduces contrastive learning to extract valuable information from both positive and negative token sequences, enhancing the quality of learned node representations. Extensive experimental results across various datasets, including homophily and heterophily graphs, demonstrate the superiority of GCFormer in node classification, when compared to representative graph neural networks (GNNs) and graph Transformers.

翻译：尽管标记化图Transformer在节点分类任务中表现出色，但其依赖相似度得分较高的有限节点子集构建标记序列的做法，忽略了其他节点的宝贵信息，从而阻碍了其充分利用图信息学习最优节点表示的能力。为克服这一局限，我们提出了一种名为GCFormer的新型图Transformer。与先前方法不同，GCFormer开发了一种混合标记生成器，可创建正负两种类型的标记序列以捕捉多样化的图信息。同时采用定制的基于Transformer的主干网络，从这些生成的标记序列中学习有意义的节点表示。此外，GCFormer引入对比学习机制，从正负标记序列中提取有价值信息，从而提升所学节点表示的质量。在包括同配图和异配图在内的多种数据集上的大量实验结果表明，与代表性的图神经网络（GNN）和图Transformer相比，GCFormer在节点分类任务中具有显著优越性。