We present GRAPE (Group Representational Position Encoding), a unified framework for positional encoding based on group actions. GRAPE unifies two families of mechanisms: (i) multiplicative rotations (Multiplicative GRAPE) in $\operatorname{SO}(d)$ and (ii) additive logit biases (Additive GRAPE) arising from unipotent actions in the general linear group $\mathrm{GL}$. In Multiplicative GRAPE, a position $n \in \mathbb{Z}$ (or $t \in \mathbb{R}$) acts as $\mathbf{G}(n) = \exp(n \, ω\, \mathbf{L})$ with a rank-2 skew-symmetric generator $\mathbf{L} \in \mathbb{R}^{d \times d}$, yielding a relative, compositional, norm-preserving map with a closed-form matrix exponential. RoPE is recovered exactly when the $d/2$ planes correspond to canonical coordinate pairs with a log-uniform spectrum. Learned commuting subspaces and compact non-commuting mixtures strictly extend this geometry to capture cross-subspace feature coupling at $O(d)$ and $O(r d)$ cost per head, respectively. In Additive GRAPE, additive logits arise from rank-1 (or low-rank) unipotent actions, recovering ALiBi and the Forgetting Transformer (FoX) as exact special cases while preserving an exact relative law and streaming cacheability. Overall, GRAPE provides a principled design space for positional geometry in long-context models, subsuming RoPE and ALiBi as special cases. Project page: https://github.com/model-architectures/GRAPE.

翻译：我们提出了GRAPE（群表示位置编码），一个基于群作用的统一位置编码框架。GRAPE统一了两类机制：(i) $\operatorname{SO}(d)$ 中的乘法旋转（乘法GRAPE）与 (ii) 由一般线性群 $\mathrm{GL}$ 中幂幺作用产生的加法对数偏置（加法GRAPE）。在乘法GRAPE中，位置 $n \in \mathbb{Z}$（或 $t \in \mathbb{R}$）通过 $\mathbf{G}(n) = \exp(n \, ω\, \mathbf{L})$ 作用，其中 $\mathbf{L} \in \mathbb{R}^{d \times d}$ 是一个秩为2的斜对称生成元，从而产生一个相对的、可组合的、保范的映射，并具有闭式矩阵指数。当 $d/2$ 个平面对应于具有对数均匀谱的规范坐标对时，可精确恢复RoPE。学习到的交换子空间与紧致的非交换混合分别以每个注意力头 $O(d)$ 和 $O(r d)$ 的代价，严格扩展了此几何结构以捕捉跨子空间的特征耦合。在加法GRAPE中，加法对数来自秩为1（或低秩）的幂幺作用，将ALiBi与遗忘Transformer（FoX）作为精确特例恢复，同时保留了精确的相对律与流式缓存能力。总体而言，GRAPE为长上下文模型中的位置几何提供了一个原则性的设计空间，将RoPE和ALiBi纳入为特例。项目页面：https://github.com/model-architectures/GRAPE。