Modern Large Language Models (LLMs) have demonstrated remarkable capabilities in complex tasks by employing search-augmented reasoning to incorporate external knowledge into long chains of thought. However, we identify a critical yet underexplored bottleneck in this paradigm, termed Knowledge Integration Decay (KID). Specifically, we observe that as the length of reasoning generated before search grows, models increasingly fail to integrate retrieved evidence into subsequent reasoning steps, limiting performance even when relevant information is available. To address this, we propose Self-Anchored Knowledge Encoding (SAKE), a training-free inference-time strategy designed to stabilize knowledge utilization. By anchoring retrieved knowledge at both the beginning and end of the reasoning process, SAKE prevents it from being overshadowed by prior context, thereby preserving its semantic integrity. Extensive experiments on multi-hop QA and complex reasoning benchmarks demonstrate that SAKE significantly mitigates KID and improves performance, offering a lightweight yet effective solution for knowledge integration in agentic LLMs.
翻译:现代大型语言模型(LLMs)通过采用搜索增强推理将外部知识融入长链思维,在复杂任务中展现出卓越能力。然而,我们发现该范式存在一个关键却未被充分探索的瓶颈,即知识整合衰减(KID)。具体而言,我们观察到随着搜索前生成推理链长度的增加,模型越来越难以将检索到的证据整合到后续推理步骤中,即使在相关信息可用时也限制了性能提升。为解决此问题,我们提出自锚定知识编码(SAKE),这是一种无需训练、在推理阶段实施的策略,旨在稳定知识利用。通过在推理过程的首尾两端锚定检索知识,SAKE可防止其被先验语境遮蔽,从而保持语义完整性。在多跳问答与复杂推理基准上的大量实验表明,SAKE能显著缓解KID现象并提升性能,为智能体化LLMs的知识整合问题提供了轻量而有效的解决方案。