Transformers are the basis of modern large language models, but relatively little is known about their precise expressive power on graphs. We study the expressive power of graph transformers (GTs) by Dwivedi and Bresson (2020) and GPS-networks by Rampásek et al. (2022), both under soft-attention and average hard-attention. Our study covers two scenarios: the theoretical setting with real numbers and the more practical case with floats. With reals, we show that in restriction to vertex properties definable in first-order logic (FO), GPS-networks have the same expressive power as graded modal logic (GML) with the global modality. With floats, GPS-networks turn out to be equally expressive as GML with the counting global modality. The latter result is absolute, not restricting to properties definable in a background logic. We also obtain similar characterizations for GTs in terms of propositional logic with the global modality (for reals) and the counting global modality (for floats).

翻译：Transformer是现代大型语言模型的基础，但关于其在图数据上的精确表达能力所知相对有限。本文研究了Dwivedi和Bresson（2020）提出的图变换器（GTs）以及Rampásek等人（2022）提出的GPS网络在软注意力与平均硬注意力机制下的表达能力。我们的研究涵盖两种场景：使用实数的理论设定和使用浮点数的实际场景。在实数情形下，我们证明当限制于一阶逻辑（FO）可定义的顶点属性时，GPS网络与带全局模态的渐变模态逻辑（GML）具有相同的表达能力。在浮点数情形下，GPS网络被证明与带计数全局模态的GML具有同等表达能力。后一结果是绝对的，不依赖于背景逻辑的可定义性限制。我们还针对图变换器获得了类似的刻画：在实数情形下对应带全局模态的命题逻辑，在浮点数情形下对应带计数全局模态的命题逻辑。