Graph Transformers (GTs) with powerful representation learning ability make a huge success in wide range of graph tasks. However, the costs behind outstanding performances of GTs are higher energy consumption and computational overhead. The complex structure and quadratic complexity during attention calculation in vanilla transformer seriously hinder its scalability on the large-scale graph data. Though existing methods have made strides in simplifying combinations among blocks or attention-learning paradigm to improve GTs' efficiency, a series of energy-saving solutions originated from biologically plausible structures are rarely taken into consideration when constructing GT framework. To this end, we propose a new spiking-based graph transformer (SGHormer). It turns full-precision embeddings into sparse and binarized spikes to reduce memory and computational costs. The spiking graph self-attention and spiking rectify blocks in SGHormer explicitly capture global structure information and recover the expressive power of spiking embeddings, respectively. In experiments, SGHormer achieves comparable performances to other full-precision GTs with extremely low computational energy consumption. The results show that SGHomer makes a remarkable progress in the field of low-energy GTs.

翻译：图Transformer（GTs）凭借其强大的表示学习能力在广泛的图任务中取得了巨大成功。然而，GTs卓越性能背后付出的代价是更高的能耗和计算开销。标准Transformer中复杂的结构以及注意力计算时的二次复杂度严重阻碍了其在大规模图数据上的可扩展性。尽管现有方法在简化模块间组合或注意力学习范式以提升GTs效率方面取得了进展，但在构建GT框架时，很少考虑源于生物合理结构的一系列节能方案。为此，我们提出了一种新型基于脉冲的图Transformer（SGHormer）。它将全精度嵌入转化为稀疏的二值化脉冲，以降低内存和计算成本。SGHormer中的脉冲图自注意力模块和脉冲修正块分别负责显式捕捉全局结构信息，并恢复脉冲嵌入的表达能力。实验表明，SGHormer以极低的计算能耗达到了与其他全精度GTs相当的性能。结果表明，SGHormer在低能耗GTs领域取得了显著进展。