While Large Language Models (LLMs) demonstrate strong performance across domains, their long-context capabilities are limited by transient neural activations causing information decay and unstructured feed-forward network (FFN) weights leading to semantic fragmentation. Inspired by the brain's working memory and cortical modularity, we propose PaceLLM, featuring two innovations: (1) a Persistent Activity (PA) Mechanism that mimics prefrontal cortex (PFC) neurons' persistent firing by introducing an activation-level memory bank to dynamically retrieve, reuse, and update critical FFN states, addressing contextual decay; and (2) Cortical Expert (CE) Clustering that emulates task-adaptive neural specialization to reorganize FFN weights into semantic modules, establishing cross-token dependencies and mitigating fragmentation. Extensive evaluations show that PaceLLM achieves 6% improvement on LongBench's Multi-document QA and 12.5-17.5% performance gains on Infinite-Bench tasks, while extending measurable context length to 200K tokens in Needle-In-A-Haystack (NIAH) tests. This work pioneers brain-inspired LLM optimization and is complementary to other works. Besides, it can be generalized to any model and enhance their long-context performance and interpretability without structural overhauls.

翻译：尽管大语言模型（LLMs）在各领域展现出强大性能，但其长上下文能力受限于瞬时神经激活导致的信息衰减，以及非结构化前馈网络（FFN）权重引发的语义碎片化。受大脑工作记忆与皮层模块化的启发，我们提出PaceLLM模型，其包含两项创新：（1）持久活动（PA）机制——通过引入激活级记忆库动态检索、复用与更新关键FFN状态，模拟前额叶皮层（PFC）神经元的持续放电特性，以应对上下文衰减；（2）皮层专家（CE）聚类——借鉴任务自适应神经特化机制，将FFN权重重组为语义模块，建立跨token依赖关系以缓解碎片化。大量实验表明，PaceLLM在LongBench的多文档问答任务上实现6%的性能提升，在Infinite-Bench任务上获得12.5-17.5%的性能增益，同时在“大海捞针”（NIAH）测试中将可测上下文长度扩展至20万token。本研究开创了类脑大语言模型优化路径，与其他工作形成互补。此外，该框架可泛化至任意模型，无需结构重构即可增强其长上下文性能与可解释性。