Modern sequence models are dominated by Transformers, where self-attention mixes information from the visible context in an input-dependent way. However, when retrieval is not sharp and attention remains diffuse over an effective support $S_{\mathrm{eff}}(t)$, the influence of any individual token is diluted, typically scaling as $O(1/S_{\mathrm{eff}}(t))$ and reaching $O(1/\ell)$ for old tokens in full-prefix settings. Structured state-space models process sequences recurrently through an explicit feedback path; selective variants such as Mamba make this feedback input-dependent, yet when freeze time cannot be sustained over long intervals, their long-range sensitivity decays exponentially with lag. Existing architectures therefore either retrieve from the past in a single read or propagate information through a single feedback chain. We introduce Sessa, a decoder that places attention inside a feedback path, enabling recurrent many-path aggregation within a layer. Under stated assumptions, Sessa admits regimes with a power-law memory tail in lag $\ell$ of order $O(\ell^{-β})$ for $0<β<1$, which is asymptotically slower than $1/\ell$; moreover, this rate is tight in an explicit diffuse uniform-routing setting where the influence is $Θ(\ell^{-β})$. Under the same conditions, only Sessa among the compared model classes realizes flexible selective retrieval, including non-decaying profiles. Empirically, under matched architectures and training budgets, Sessa achieves the strongest performance on our long-context benchmarks while remaining competitive with Transformer and Mamba style baselines on short-context language modeling.

翻译：现代序列模型以Transformer为主导，其中自注意力机制以输入依赖的方式从可见上下文中混合信息。然而，当检索不够精准且注意力在有效支持域$S_{\mathrm{eff}}(t)$上保持弥散时，单个token的影响力会被稀释，典型缩放关系为$O(1/S_{\mathrm{eff}}(t))$，在全前缀设置中对于旧token可达$O(1/ℓ)$。结构化状态空间模型通过显式反馈路径递归处理序列；诸如Mamba等选择性变体使该反馈具有输入依赖性，但当冻结时间无法长期维持时，其长程敏感性会随滞后量呈指数衰减。现有架构要么以单次读取方式从历史中检索信息，要么通过单一反馈链传播信息。我们提出Sessa——一种将注意力机制嵌入反馈路径的解码器，能够在单层内实现递归多路径聚合。在既定假设下，Sessa可实现在滞后ℓ上幂律记忆尾部的机制，阶数为$O(ℓ^{-β})$（$0<β<1$），该衰减速率渐近慢于$1/ℓ$；且在显式弥散均匀路由设置中该速率具有紧致性，影响力精确为$Θ(ℓ^{-β})$。在相同条件下，Sessa是唯一能够实现灵活选择性检索（包括非衰减模式）的模型类别。在匹配架构和训练预算的条件下，Sessa在长上下文基准测试中取得了最优性能，同时在短上下文语言建模中与Transformer和Mamba风格基线保持竞争力。