Graph Transformers (GTs) have demonstrated remarkable performance in incorporating various graph structure information, e.g., long-range structural dependency, into graph representation learning. However, self-attention -- the core module of GTs -- preserves only low-frequency signals on graph features, retaining only homophilic patterns that capture similar features among the connected nodes. Consequently, it has insufficient capacity in modeling complex node label patterns, such as the opposite of homophilic patterns -- heterophilic patterns. Some improved GTs deal with the problem by learning polynomial filters or performing self-attention over the first-order graph spectrum. However, these GTs either ignore rich information contained in the whole spectrum or neglect higher-order spectrum information, resulting in limited flexibility and frequency response in their spectral filters. To tackle these challenges, we propose a novel GT network, namely Graph Fourier Kolmogorov-Arnold Transformers (GrokFormer), to go beyond the self-attention in GTs. GrokFormer leverages learnable activation functions in order-$K$ graph spectrum through Fourier series modeling to i) learn eigenvalue-targeted filter functions producing learnable base that can capture a broad range of frequency signals flexibly, and ii) extract first- and higher-order graph spectral information adaptively. In doing so, GrokFormer can effectively capture intricate patterns hidden across different orders and levels of frequency signals, learning expressive, order-and-frequency-adaptive graph representations. Comprehensive experiments conducted on 10 node classification datasets across various domains, scales, and levels of graph heterophily, as well as 5 graph classification datasets, demonstrate that GrokFormer outperforms state-of-the-art GTs and other advanced graph neural networks.

翻译：图变换器（GTs）在将各种图结构信息（例如长程结构依赖）融入图表示学习方面已展现出卓越性能。然而，自注意力机制——GTs的核心模块——仅保留图特征上的低频信号，仅捕获相连节点间相似特征的同配模式。因此，其在建模复杂节点标签模式（如与同配模式相反的异配模式）方面能力不足。一些改进的GTs通过学习多项式滤波器或在图的一阶谱上执行自注意力来处理该问题。然而，这些GTs要么忽略了整个频谱中包含的丰富信息，要么忽视了高阶谱信息，导致其谱滤波器的灵活性和频率响应有限。为应对这些挑战，我们提出了一种新颖的GT网络，即图傅里叶-柯尔莫哥洛夫-阿诺德变换器（GrokFormer），以超越GTs中的自注意力机制。GrokFormer通过傅里叶级数建模，在$K$阶图谱中利用可学习的激活函数，以：i）学习特征值导向的滤波器函数，生成可灵活捕获广泛频率信号的可学习基；ii）自适应地提取一阶及高阶图谱信息。通过这种方式，GrokFormer能够有效捕获隐藏在不同阶次和频率信号水平下的复杂模式，学习具有表达力、阶次与频率自适应的图表示。在涵盖不同领域、规模和图异配性水平的10个节点分类数据集以及5个图分类数据集上进行的综合实验表明，GrokFormer优于最先进的GTs及其他先进的图神经网络。