Recently, transformer architectures for graphs emerged as an alternative to established techniques for machine learning with graphs, such as graph neural networks. So far, they have shown promising empirical results, e.g., on molecular prediction datasets, often attributed to their ability to circumvent graph neural networks' shortcomings, such as over-smoothing and over-squashing. Here, we derive a taxonomy of graph transformer architectures, bringing some order to this emerging field. We overview their theoretical properties, survey structural and positional encodings, and discuss extensions for important graph classes, e.g., 3D molecular graphs. Empirically, we probe how well graph transformers can recover various graph properties, how well they can deal with heterophilic graphs, and to what extent they prevent over-squashing. Further, we outline open challenges and research direction to stimulate future work. Our code is available at https://github.com/luis-mueller/probing-graph-transformers.

翻译：近期，面向图的Transformer架构作为图神经网络等图机器学习成熟技术的替代方案崭露头角。目前，这些架构已在分子预测数据集等任务上展现出令人瞩目的实证成果，这通常归因于其能够规避图神经网络的固有缺陷，例如过平滑与过挤压问题。本文构建了图Transformer架构的分类体系，为该新兴领域梳理出系统性框架。我们综述了其理论特性，调研了结构编码与位置编码方法，并讨论了面向重要图类型（如3D分子图）的扩展方案。在实证层面，我们探究了图Transformer恢复多种图属性的能力、处理异配图的效能，以及预防过挤压问题的程度。此外，我们指出了开放挑战与研究方向以推动未来工作。相关代码已开源：https://github.com/luis-mueller/probing-graph-transformers。