Linear attention mechanisms have emerged as efficient alternatives to full self-attention in Graph Transformers, offering linear time complexity. However, existing linear attention models often suffer from a significant drop in expressiveness due to low-rank projection structures and overly uniform attention distributions. We theoretically prove that these properties reduce the class separability of node representations, limiting the model's classification ability. To address this, we propose a novel hybrid framework that enhances both the rank and focus of attention. Specifically, we enhance linear attention by attaching a gated local graph network branch to the value matrix, thereby increasing the rank of the resulting attention map. Furthermore, to alleviate the excessive smoothing effect inherent in linear attention, we introduce a learnable log-power function into the attention scores to reduce entropy and sharpen focus. We theoretically show that this function decreases entropy in the attention distribution, enhancing the separability of learned embeddings. Extensive experiments on both homophilic and heterophilic graph benchmarks demonstrate that our method achieves competitive performance while preserving the scalability of linear attention.

翻译：线性注意力机制已成为图Transformer中全自注意力机制的高效替代方案，其具备线性时间复杂度。然而，现有的线性注意力模型常因低秩投影结构和过于均匀的注意力分布而导致表达能力显著下降。我们从理论上证明，这些特性会降低节点表示的类间可分性，从而限制模型的分类能力。为解决这一问题，我们提出了一种新颖的混合框架，该框架同时增强了注意力的秩与聚焦能力。具体而言，我们通过在值矩阵上附加一个门控局部图网络分支来增强线性注意力，从而提升所得注意力图的秩。此外，为缓解线性注意力固有的过度平滑效应，我们在注意力分数中引入一个可学习的对数幂函数，以降低熵值并锐化聚焦。我们从理论上证明，该函数能够降低注意力分布的熵，从而增强所学嵌入的可分性。在同配性与异配性图基准上的大量实验表明，我们的方法在保持线性注意力可扩展性的同时，实现了具有竞争力的性能。