Recent studies have highlighted the limitations of message-passing based graph neural networks (GNNs), e.g., limited model expressiveness, over-smoothing, over-squashing, etc. To alleviate these issues, Graph Transformers (GTs) have been proposed which work in the paradigm that allows message passing to a larger coverage even across the whole graph. Hinging on the global range attention mechanism, GTs have shown a superpower for representation learning on homogeneous graphs. However, the investigation of GTs on heterogeneous information networks (HINs) is still under-exploited. In particular, on account of the existence of heterogeneity, HINs show distinct data characteristics and thus require different treatment. To bridge this gap, in this paper we investigate the representation learning on HINs with Graph Transformer, and propose a novel model named HINormer, which capitalizes on a larger-range aggregation mechanism for node representation learning. In particular, assisted by two major modules, i.e., a local structure encoder and a heterogeneous relation encoder, HINormer can capture both the structural and heterogeneous information of nodes on HINs for comprehensive node representations. We conduct extensive experiments on four HIN benchmark datasets, which demonstrate that our proposed model can outperform the state-of-the-art.

翻译：近期的研究揭示了基于消息传递的图神经网络（GNN）的局限性，例如模型表达能力有限、过度平滑、过度压缩等问题。为缓解这些问题，研究者提出了图Transformer（GT），其工作范式中允许消息传递覆盖更大范围，甚至跨越整个图。依托全局范围的注意力机制，GT在同质图表示学习上展现出卓越能力。然而，GT在异质信息网络（HIN）上的研究仍显不足。具体而言，由于异质性的存在，HIN呈现出独特的数据特征，因此需要区别对待。为弥补这一空白，本文研究了基于图Transformer的异质信息网络表示学习，并提出一种名为HINormer的新型模型。该模型利用更大范围的聚合机制进行节点表示学习。特别地，借助两个核心模块——局部结构编码器和异质关系编码器，HINormer能够捕获HIN上节点的结构与异质信息，从而获得全面的节点表示。我们在四个HIN基准数据集上进行了广泛实验，结果表明所提模型能够超越现有最先进方法。