Most current long-context language models still rely on attention to handle both local interaction and long-range state, which leaves relatively little room to test alternative decompositions of sequence modeling. We propose LPC-SM, a hybrid autoregressive architecture that separates local attention, persistent memory, predictive correction, and run-time control within the same block, and we use Orthogonal Novelty Transport (ONT) to govern slow-memory writes. We evaluate a 158M-parameter model in three stages spanning base language modeling, mathematical continuation, and 4096-token continuation. Removing mHC raises the Stage-A final LM loss from 12.630 to 15.127, while adaptive sparse control improves the Stage-B final LM loss from 12.137 to 10.787 relative to a matched fixed-ratio continuation. The full route remains stable at sequence length 4096, where Stage C ends with final LM loss 11.582 and improves the delayed-identifier diagnostic from 14.396 to 12.031 in key cross-entropy. Taken together, these results show that long-context autoregressive modeling can be organized around a broader division of labor than attention alone.

翻译：当前多数长上下文语言模型仍依赖注意力机制同时处理局部交互与长程状态，导致序列建模的替代分解方案研究空间有限。我们提出LPC-SM——一种在同一模块内分离局部注意力、持久记忆、预测校正与运行时控制的混合自回归架构，并采用正交新颖性传输机制管控慢速记忆写入。我们通过基础语言建模、数学延续与4096词元延续三阶段评估158M参数模型。移除mHC使A阶段最终语言模型损失从12.630升至15.127，而自适应稀疏控制相较匹配固定比率延续，将B阶段最终语言模型损失从12.137降至10.787。完整路径在序列长度4096下保持稳定，C阶段最终语言模型损失为11.582，并将关键交叉熵延迟标识诊断指标从14.396优化至12.031。综合结果表明，长上下文自回归建模可通过比单一注意力更广泛的功能划分来组织。