Large Language Models face challenges in long-horizon agentic tasks as their constrained memory is easily overwhelmed by distracting or irrelevant context. Existing working memory methods typically rely on external, heuristic mechanisms that are decoupled from the agent's core policy. In this work, we reframe working memory management as a learnable, intrinsic capability. We propose a novel framework, Memory-as-Action, where an agent actively manages its working memory by executing explicit editing operations as part of a unified policy. This formulation allows an agent, trained via reinforcement learning, to balance memory curation against long-term task objectives under given resource constraints. However, such memory editing actions break the standard assumption of a continuously growing prefix in LLM interactions, leading to what we call trajectory fractures. These non-prefix changes disrupt the causal continuity required by standard policy gradient methods, making those methods inapplicable. To address this, we propose a new algorithm, Dynamic Context Policy Optimization, which enables stable end-to-end reinforcement learning by segmenting trajectories at memory action points and applying trajectory-level advantages to the resulting action segments. Our results demonstrate that jointly optimizing for task reasoning and memory management in an end-to-end fashion not only reduces overall computational consumption but also improves task performance, driven by adaptive context curation strategies tailored to the model's intrinsic capabilities.

翻译：大型语言模型在执行长程智能体任务时面临挑战，因其有限的内存容量极易被干扰性或无关上下文所淹没。现有工作记忆方法通常依赖与智能体核心策略解耦的外部启发式机制。本研究将工作记忆管理重新定义为一种可学习的内部能力。我们提出了一种新颖的框架——记忆即行动，在该框架中，智能体通过执行显式编辑操作作为统一策略的组成部分，主动管理其工作记忆。这种形式化方法使得通过强化学习训练的智能体能够在给定资源约束下，平衡记忆管理与长期任务目标。然而，此类记忆编辑行为打破了LLM交互中上下文前缀持续增长的标准假设，导致了我们称之为轨迹断裂的现象。这些非前缀变更破坏了标准策略梯度方法所需的因果连续性，使得传统方法不再适用。为解决此问题，我们提出了一种新算法——动态上下文策略优化，该算法通过在记忆操作点对轨迹进行分段，并将轨迹级优势应用于生成的动作片段，实现了稳定的端到端强化学习。实验结果表明，以端到端方式联合优化任务推理与记忆管理，不仅能降低整体计算消耗，还能通过适配模型内部能力的自适应上下文管理策略提升任务性能。