Graph representation learning (GRL) methods, such as graph neural networks and graph transformer models, have been successfully used to analyze graph-structured data, mainly focusing on node classification and link prediction tasks. However, the existing studies mostly only consider local connectivity while ignoring long-range connectivity and the roles of nodes. In this paper, we propose Unified Graph Transformer Networks (UGT) that effectively integrate local and global structural information into fixed-length vector representations. First, UGT learns local structure by identifying the local substructures and aggregating features of the $k$-hop neighborhoods of each node. Second, we construct virtual edges, bridging distant nodes with structural similarity to capture the long-range dependencies. Third, UGT learns unified representations through self-attention, encoding structural distance and $p$-step transition probability between node pairs. Furthermore, we propose a self-supervised learning task that effectively learns transition probability to fuse local and global structural features, which could then be transferred to other downstream tasks. Experimental results on real-world benchmark datasets over various downstream tasks showed that UGT significantly outperformed baselines that consist of state-of-the-art models. In addition, UGT reaches the expressive power of the third-order Weisfeiler-Lehman isomorphism test (3d-WL) in distinguishing non-isomorphic graph pairs. The source code is available at https://github.com/NSLab-CUK/Unified-Graph-Transformer.

翻译：图表示学习方法（如图神经网络和图Transformer模型）已成功用于分析图结构数据，主要聚焦于节点分类和链路预测任务。然而，现有研究大多仅考虑局部连通性，而忽略了长程连通性和节点角色。本文提出统一图Transformer网络（UGT），将局部和全局结构信息有效整合为固定长度向量表示。首先，UGT通过识别局部子结构并聚合每个节点$k$跳邻域的特征来学习局部结构。其次，我们构建虚拟边以桥接结构相似但距离较远的节点，从而捕获长程依赖关系。再次，UGT通过自注意力机制学习统一表示，编码节点对之间的结构距离和$p$步转移概率。此外，我们提出一种自监督学习任务，有效学习转移概率以融合局部和全局结构特征，该特征可迁移至其他下游任务。在真实世界基准数据集上的各类下游任务实验结果表明，UGT显著优于包含最先进模型的基线方法。同时，UGT在区分非同构图对时达到了三阶Weisfeiler-Lehman同构测试（3d-WL）的表达能力。源代码已开源至https://github.com/NSLab-CUK/Unified-Graph-Transformer。