A current goal in the graph neural network literature is to enable transformers to operate on graph-structured data, given their success on language and vision tasks. Since the transformer's original sinusoidal positional encodings (PEs) are not applicable to graphs, recent work has focused on developing graph PEs, rooted in spectral graph theory or various spatial features of a graph. In this work, we introduce a new graph PE, Graph Automaton PE (GAPE), based on weighted graph-walking automata (a novel extension of graph-walking automata). We compare the performance of GAPE with other PE schemes on both machine translation and graph-structured tasks, and we show that it generalizes several other PEs. An additional contribution of this study is a theoretical and controlled experimental comparison of many recent PEs in graph transformers, independent of the use of edge features.

翻译：当前图神经网络领域的一个目标是使Transformer能够处理图结构数据，因其在语言和视觉任务中取得了成功。由于Transformer原本的正弦位置编码（PEs）不适用于图结构，近期工作聚焦于开发基于谱图理论或图的各种空间特征的图位置编码。本文提出一种基于加权图游走自动机（图游走自动机的新型扩展）的新型图位置编码——图自动机位置编码（GAPE）。我们将GAPE与其他位置编码方案在机器翻译和图结构任务上的性能进行了比较，并证明它能够泛化多种其他位置编码。本研究的另一项贡献是，在不依赖边特征的情况下，对图Transformer中多种近期位置编码进行了理论分析和受控实验比较。