Large Language Models (LLMs) face significant challenges in long-context processing, including quadratic computational costs, information forgetting, and the context fragmentation inherent in retrieval-augmented generation (RAG). We propose a cognitively inspired framework for efficient long-context inference based on chunk-wise compression and selective memory recall, rather than processing all raw tokens. The framework segments long inputs into chunks and encodes each chunk into compressed memory representations using a learned compressor. A gating module dynamically selects relevant memory blocks, which are then iteratively processed by a reasoning module with an evolving working memory to solve downstream tasks. The compressor and reasoner are jointly optimized via end-to-end reinforcement learning, while the gating module is trained separately as a classifier. Experimental results show that the proposed method achieves competitive accuracy on multi-hop reasoning benchmarks such as RULER-HQA, extrapolates context length from 7K to 1.75M tokens, and offers a favorable accuracy-efficiency trade-off compared to strong long-context baselines. In particular, it achieves up to a 2 times reduction in peak GPU memory usage and a 6 times inference speedup over MemAgent.

翻译：大型语言模型（LLMs）在长上下文处理中面临显著挑战，包括二次计算成本、信息遗忘以及检索增强生成（RAG）固有的上下文碎片化问题。我们提出一种受认知启发的框架，用于实现高效的长上下文推理，其核心在于基于分块压缩与选择性记忆检索，而非处理所有原始标记。该框架将长输入分割为多个块，并使用一个学习到的压缩器将每个块编码为压缩的记忆表示。一个门控模块动态选择相关的记忆块，随后由推理模块结合不断演化的短期记忆进行迭代处理，以解决下游任务。压缩器与推理器通过端到端强化学习进行联合优化，而门控模块则作为分类器单独训练。实验结果表明，所提方法在RULER-HQA等多跳推理基准上取得了具有竞争力的准确率，能够将上下文长度从7K标记外推至1.75M标记，并且与强大的长上下文基线模型相比，提供了更优的准确率-效率权衡。特别地，与MemAgent相比，该方法实现了高达2倍的峰值GPU内存使用降低和6倍的推理加速。