Graph transformers need strong inductive biases to derive meaningful attention scores. Yet, current proposals rarely address methods capturing longer ranges, hierarchical structures, or community structures, as they appear in various graphs such as molecules, social networks, and citation networks. In this paper, we propose a hierarchy-distance structural encoding (HDSE), which models a hierarchical distance between the nodes in a graph focusing on its multi-level, hierarchical nature. In particular, this yields a framework which can be flexibly integrated with existing graph transformers, allowing for simultaneous application with other positional representations. Through extensive experiments on 12 real-world datasets, we demonstrate that our HDSE method successfully enhances various types of baseline transformers, achieving state-of-the-art empirical performances on 10 benchmark datasets.

翻译：图神经网络中的Transformer需要强大的归纳偏置来生成有意义的注意力分数。然而，现有方案很少涉及解决长程依赖、层级结构或社区结构的捕获方法，而这些结构广泛存在于分子、社交网络和引文网络等各类图中。本文提出一种层级距离结构编码（HDSE），该编码通过聚焦图的多级层级特性来建模节点间的层级距离。具体而言，这一框架可灵活集成至现有图Transformer中，并允许与其他位置表示方法同时使用。通过在12个真实世界数据集上的大量实验，我们证明HDSE方法能有效增强多种基线Transformer模型，并在10个基准数据集上实现了最先进的实证性能。