Large Language Models (LLMs) often exhibit increased response latency and degraded answer quality as dialogue length grows, making effective context management essential. However, existing methods rely on extra LLM calls to build memory or perform offline memory construction without considering the current user utterance, which can introduce inefficiencies or disrupt conversational continuity. We introduce DyCP, a lightweight context management method that dynamically segment and retrieve relevant memory at query time. It preserves the sequential structure of dialogue without predefined topic boundaries and supports efficient, adaptive context retrieval. Across three long-form dialogue benchmarks, LoCoMo, MT-Bench+, and SCM4LLMs, and multiple LLMs, DyCP consistently improves answer quality while reducing response latency. We also examine the gap between modern LLMs' expanded context windows and their actual long-context processing capacity, highlighting the continued importance of effective context management.

翻译：随着对话长度增加，大语言模型（LLMs）通常表现出响应延迟上升和回答质量下降的问题，这使得有效的上下文管理至关重要。然而，现有方法依赖额外的大语言模型调用来构建记忆，或在不考虑当前用户话语的情况下进行离线记忆构建，这可能引入低效或破坏对话连续性。我们提出DyCP，一种轻量级的上下文管理方法，能够在查询时动态分割并检索相关记忆。该方法无需预定义主题边界即可保持对话的序列结构，并支持高效、自适应的上下文检索。在三个长对话基准测试（LoCoMo、MT-Bench+和SCM4LLMs）以及多个大语言模型上的实验表明，DyCP在降低响应延迟的同时持续提升了回答质量。我们还探讨了现代大语言模型扩展的上下文窗口与其实际长上下文处理能力之间的差距，强调了有效上下文管理的持续重要性。