The dominant paradigm for machine learning on graphs uses Message Passing Graph Neural Networks (MP-GNNs), in which node representations are updated by aggregating information in their local neighborhood. Recently, there have been increasingly more attempts to adapt the Transformer architecture to graphs in an effort to solve some known limitations of MP-GNN. A challenging aspect of designing Graph Transformers is integrating the arbitrary graph structure into the architecture. We propose Graph Diffuser (GD) to address this challenge. GD learns to extract structural and positional relationships between distant nodes in the graph, which it then uses to direct the Transformer's attention and node representation. We demonstrate that existing GNNs and Graph Transformers struggle to capture long-range interactions and how Graph Diffuser does so while admitting intuitive visualizations. Experiments on eight benchmarks show Graph Diffuser to be a highly competitive model, outperforming the state-of-the-art in a diverse set of domains.

翻译：图上的机器学习主流范式是消息传递图神经网络（MP-GNN），其通过聚合节点局部邻域的信息来更新节点表征。近年来，越来越多的工作尝试将Transformer架构适配到图结构上，以解决MP-GNN的一些已知局限性。设计图Transformer的一个挑战性在于将任意的图结构集成到架构中。我们提出图扩散器（GD）来应对这一挑战。GD学习提取图中远距离节点之间的结构和位置关系，并用其指导Transformer的注意力机制和节点表征。我们证明现有GNN和图Transformer难以捕获长程交互，而图扩散器不仅能实现这一目标，还能提供直观的可视化结果。在八个基准上的实验表明，图扩散器是一个极具竞争力的模型，在多个不同领域都超越了当前最先进水平。