We present GRAPE (Group Representational Position Encoding), a unified framework for positional encoding based on group actions. GRAPE unifies two families of mechanisms: (i) multiplicative rotations (Multiplicative GRAPE) in $\operatorname{SO}(d)$ and (ii) additive logit biases (Additive GRAPE) arising from unipotent actions in the general linear group $\mathrm{GL}$. In Multiplicative GRAPE, a position $n \in \mathbb{Z}$ (or $t \in \mathbb{R}$) acts as $\mathbf{G}(n) = \exp(n \, ω\, \mathbf{L})$ with a rank-2 skew-symmetric generator $\mathbf{L} \in \mathbb{R}^{d \times d}$, yielding a relative, compositional, norm-preserving map with a closed-form matrix exponential. RoPE is recovered exactly when the $d/2$ planes correspond to canonical coordinate pairs with a log-uniform spectrum. Learned commuting subspaces and compact non-commuting mixtures strictly extend this geometry to capture cross-subspace feature coupling at $O(d)$ and $O(r d)$ cost per head, respectively. In Additive GRAPE, additive logits arise from rank-1 (or low-rank) unipotent actions, recovering ALiBi and the Forgetting Transformer (FoX) as exact special cases while preserving an exact relative law and streaming cacheability. Overall, GRAPE provides a principled design space for positional geometry in long-context models, subsuming RoPE and ALiBi as special cases. Project page: https://github.com/model-architectures/GRAPE.

翻译：我们提出了GRAPE（群表示位置编码），一种基于群作用的统一位置编码框架。GRAPE统一了两类机制：（i）$\operatorname{SO}(d)$中的乘法旋转（乘法GRAPE），以及（ii）由一般线性群$\mathrm{GL}$中的幂幺作用产生的加法对数偏置（加法GRAPE）。在乘法GRAPE中，位置$n \in \mathbb{Z}$（或$t \in \mathbb{R}$）通过$\mathbf{G}(n) = \exp(n \, ω\, \mathbf{L})$作用，其中$\mathbf{L} \in \mathbb{R}^{d \times d}$为秩2斜对称生成元，产生一个具有闭式矩阵指数的相对、复合、保范映射。当$d/2$个平面对应于具有对数均匀谱的规范坐标对时，可精确恢复RoPE。学习的可交换子空间和紧致非可交换混合分别以每头$O(d)$和$O(r d)$的代价严格扩展此几何结构，以捕捉跨子空间特征耦合。在加法GRAPE中，加法对数来自秩1（或低秩）幂幺作用，精确恢复ALiBi和遗忘Transformer（FoX）作为特例，同时保持精确的相对定律和流式缓存能力。总体而言，GRAPE为长上下文模型中的位置几何提供了一个原则性的设计空间，将RoPE和ALiBi作为特例包含其中。项目页面：https://github.com/model-architectures/GRAPE。