Graph Transformers (GTs), which simultaneously integrate message-passing and self-attention mechanisms, have achieved promising empirical results in some graph prediction tasks. Although these approaches show the potential of Transformers in capturing long-range graph topology information, issues concerning the quadratic complexity and high computing energy consumption severely limit the scalability of GTs on large-scale graphs. Recently, as brain-inspired neural networks, Spiking Neural Networks (SNNs), facilitate the development of graph representation learning methods with lower computational and storage overhead through the unique event-driven spiking neurons. Inspired by these characteristics, we propose a linear-time Graph Transformer using Spiking Vector Quantization (GT-SVQ) for node classification. GT-SVQ reconstructs codebooks based on rate coding outputs from spiking neurons, and injects the codebooks into self-attention blocks to aggregate global information in linear complexity. Besides, spiking vector quantization effectively alleviates codebook collapse and the reliance on complex machinery (distance measure, auxiliary loss, etc.) present in previous vector quantization-based graph learning methods. In experiments, we compare GT-SVQ with other state-of-the-art baselines on node classification datasets ranging from small to large. Experimental results show that GT-SVQ has achieved competitive performances on most datasets while maintaining up to 130x faster inference speed compared to other GTs.

翻译：图Transformer（GTs）通过同时整合消息传递与自注意力机制，已在部分图预测任务中取得了良好的实证结果。尽管这些方法展现了Transformer在捕获长程图拓扑信息方面的潜力，但其二次复杂度与高计算能耗问题严重制约了GTs在大规模图上的可扩展性。近年来，作为类脑神经网络的脉冲神经网络（SNNs）凭借其独特的事件驱动脉冲神经元特性，推动了具有更低计算与存储开销的图表示学习方法的发展。受此启发，我们提出一种基于脉冲向量量化的线性时间图Transformer（GT-SVQ）用于节点分类。GT-SVQ基于脉冲神经元的脉冲频率编码输出重构码本，并将码本注入自注意力模块中以线性复杂度聚合全局信息。此外，脉冲向量量化有效缓解了码本坍缩问题，并降低了对既往基于向量量化的图学习方法中复杂机制（距离度量、辅助损失函数等）的依赖。在实验中，我们将GT-SVQ与其他前沿基线方法在从小型到大型的节点分类数据集上进行比较。实验结果表明，GT-SVQ在多数数据集上取得了具有竞争力的性能，同时相比其他GTs实现了最高达130倍的推理加速。