Large Language Models (LLMs) have shown remarkable performance in multi-turn dialogue. However, in multi-turn dialogue, models still struggle to stay aligned with what has been established earlier, follow dependencies across many turns, and avoid drifting into incorrect facts as the interaction grows longer. Existing approaches primarily focus on extending the context window, introducing external memory, or applying context compression, yet these methods still face limitations such as \textbf{contextual inertia} and \textbf{state drift}. To address these challenges, we propose the \textbf{A}daptive \textbf{C}ontext \textbf{R}efactoring \textbf{(ACR)} Framework, which dynamically monitors and reshapes the interaction history to mitigate contextual inertia and state drift actively. ACR is built on a library of context refactoring operators and a teacher-guided self-evolving training paradigm that learns when to intervene and how to refactor, thereby decoupling context management from the reasoning process. Extensive experiments on multi-turn dialogue demonstrate that our method significantly outperforms existing baselines while reducing token consumption.
翻译:大语言模型(LLM)在多轮对话中已展现出卓越的性能。然而,在多轮对话中,模型仍难以持续保持与先前已建立内容的一致性、遵循跨越多个轮次的依赖关系,并避免随着交互增长而偏离至错误事实。现有方法主要集中于扩展上下文窗口、引入外部记忆或应用上下文压缩,但这些方法仍面临**上下文惯性**和**状态漂移**等局限。为应对这些挑战,我们提出了**自适应上下文重构(ACR)框架**,该框架动态监控并重塑交互历史,以主动缓解上下文惯性与状态漂移。ACR建立在一个上下文重构算子库以及一种教师引导的自进化训练范式之上,该范式学习何时进行干预以及如何重构,从而将上下文管理与推理过程解耦。在多轮对话上的大量实验表明,我们的方法显著优于现有基线,同时降低了令牌消耗。