Adapting transformer positional encoding to meshes and graph-structured data presents significant computational challenges: exact spectral methods require cubic-complexity eigendecomposition and can inadvertently break gauge invariance through numerical solver artifacts, while efficient approximate methods sacrifice gauge symmetry by design. Both failure modes cause catastrophic generalization in inductive learning, where models trained with one set of numerical choices fail when encountering different spectral decompositions of similar graphs or discretizations of the same mesh. We propose GIST (Gauge-Invariant Spectral Transformers), a new graph transformer architecture that resolves this challenge by achieving end-to-end $\mathcal{O}(N)$ complexity through random projections while algorithmically preserving gauge invariance via inner-product-based attention on the projected embeddings. We prove GIST achieves discretization-invariant learning with bounded mismatch error, enabling parameter transfer across arbitrary mesh resolutions for neural operator applications. Empirically, GIST matches state-of-the-art on standard graph benchmarks (e.g., achieving 99.50% micro-F1 on PPI) while uniquely scaling to mesh-based Neural Operator benchmarks with up to 750K nodes, achieving state-of-the-art aerodynamic prediction on the challenging DrivAerNet and DrivAerNet++ datasets.

翻译：将Transformer位置编码应用于网格和图结构数据面临显著的计算挑战：精确谱方法需要立方复杂度的特征分解，且可能因数值求解器伪影无意中破坏规范不变性；而高效的近似方法则通过设计牺牲了规范对称性。这两种失效模式都会导致归纳学习中的灾难性泛化问题，即使用一组数值选择训练的模型在遇到相似图的不同谱分解或同一网格的不同离散化时会失效。我们提出GIST（规范不变谱变换器），这是一种新的图Transformer架构，通过随机投影实现端到端$\mathcal{O}(N)$复杂度，同时在算法上通过基于投影嵌入内积的注意力机制保持规范不变性，从而解决了这一挑战。我们证明GIST能以有界失配误差实现离散化不变学习，使得神经算子应用中的参数能够跨任意网格分辨率迁移。实证表明，GIST在标准图基准测试中达到最先进水平（如在PPI数据集上实现99.50%的微平均F1分数），同时能独特地扩展到包含多达75万个节点的基于网格的神经算子基准测试，在具有挑战性的DrivAerNet和DrivAerNet++数据集上实现了最先进的空气动力学预测性能。