The transformer architecture has shown remarkable success in various domains, such as natural language processing and computer vision. When it comes to graph learning, transformers are required not only to capture the interactions between pairs of nodes but also to preserve graph structures connoting the underlying relations and proximity between them, showing the expressive power to capture different graph structures. Accordingly, various structure-preserving graph transformers have been proposed and widely used for various tasks, such as graph-level tasks in bioinformatics and chemoinformatics. However, strategies related to graph structure preservation have not been well organized and systematized in the literature. In this paper, we provide a comprehensive overview of structure-preserving graph transformers and generalize these methods from the perspective of their design objective. First, we divide strategies into four main groups: node feature modulation, context node sampling, graph rewriting, and transformer architecture improvements. We then further divide the strategies according to the coverage and goals of graph structure preservation. Furthermore, we also discuss challenges and future directions for graph transformer models to preserve the graph structure and understand the nature of graphs.

翻译：Transformer架构在自然语言处理与计算机视觉等领域已展现出显著成功。当应用于图学习时，Transformer不仅需要捕捉节点间的交互作用，还需保留蕴含底层关系与邻近性的图结构，从而具备捕获不同图结构的表达能力。据此，各类结构保持图Transformer被提出并广泛应用于生物信息学与化学信息学中的图级任务等场景。然而，关于图结构保持策略的系统性整理与归纳在现有文献中尚不充分。本文全面梳理了结构保持图Transformer的研究进展，并从设计目标视角对这些方法进行概括。首先，我们将策略划分为四大类：节点特征调制、上下文节点采样、图重写与Transformer架构改进。继而根据图结构保持的覆盖范围与目标对策略进行进一步细分。此外，我们还探讨了图Transformer模型在保持图结构与理解图本质方面面临的挑战与未来发展方向。