While Large Language Models (LLMs) demonstrate strong performance across domains, their long-context capabilities are limited by transient neural activations causing information decay and unstructured feed-forward network (FFN) weights leading to semantic fragmentation. Inspired by the brain's working memory and cortical modularity, we propose PaceLLM, featuring two innovations: (1) a Persistent Activity (PA) Mechanism that mimics prefrontal cortex (PFC) neurons' persistent firing by introducing an activation-level memory bank to dynamically retrieve, reuse, and update critical FFN states, addressing contextual decay; and (2) Cortical Expert (CE) Clustering that emulates task-adaptive neural specialization to reorganize FFN weights into semantic modules, establishing cross-token dependencies and mitigating fragmentation. Extensive evaluations show that PaceLLM achieves 6% improvement on LongBench's Multi-document QA and 12.5-17.5% performance gains on Infinite-Bench tasks, while extending measurable context length to 200K tokens in Needle-In-A-Haystack (NIAH) tests. This work pioneers brain-inspired LLM optimization and is complementary to other works. Besides, it can be generalized to any model and enhance their long-context performance and interpretability without structural overhauls.

翻译：尽管大语言模型（LLMs）在各领域展现出强大性能，但其长上下文能力受限于两个问题：瞬态神经激活导致信息衰减，以及非结构化前馈网络（FFN）权重引发语义碎片化。受大脑工作记忆与皮层模块化特性启发，我们提出PaceLLM，包含两项创新：（1）持久活动（PA）机制——通过模拟前额叶皮层（PFC）神经元的持续性放电，引入激活级记忆库动态检索、复用并更新关键FFN状态，以解决上下文衰减问题；（2）皮层专家（CE）聚类——模拟任务自适应神经特化机制，将FFN权重重组为语义模块，建立跨令牌依赖关系并缓解碎片化。大量评估表明，PaceLLM在LongBench多文档问答任务上提升6%，在Infinite-Bench任务中实现12.5%-17.5%的性能增益，并在“大海捞针”（NIAH）测试中可将可测量上下文长度扩展至200K tokens。本研究开创了脑启发式LLM优化路径，与其他工作互补，且可泛化至任意模型，无需结构性改造即可增强其长上下文性能与可解释性。