Large language model (LLM) agents face fundamental limitations in long-horizon reasoning due to finite context windows, making effective memory management critical. Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components, relying on heuristics or auxiliary controllers, which limits adaptability and end-to-end optimization. In this paper, we propose Agentic Memory (AgeMem), a unified framework that integrates LTM and STM management directly into the agent's policy. AgeMem exposes memory operations as tool-based actions, enabling the LLM agent to autonomously decide what and when to store, retrieve, update, summarize, or discard information. To train such unified behaviors, we propose a three-stage progressive reinforcement learning strategy and design a step-wise GRPO to address sparse and discontinuous rewards induced by memory operations. Experiments on five long-horizon benchmarks demonstrate that AgeMem consistently outperforms strong memory-augmented baselines across multiple LLM backbones, achieving improved task performance, higher-quality long-term memory, and more efficient context usage.

翻译：大语言模型（LLM）智能体因受限于有限的上下文窗口，在长程推理中存在根本性制约，这使得有效的记忆管理至关重要。现有方法通常将长期记忆（LTM）与短期记忆（STM）作为独立组件处理，依赖于启发式规则或辅助控制器，这限制了其适应性与端到端优化的潜力。本文提出代理记忆（AgeMem），一个将LTM与STM管理直接集成到智能体策略中的统一框架。AgeMem将记忆操作暴露为基于工具的动作，使LLM智能体能够自主决定存储、检索、更新、总结或丢弃何种信息以及何时执行这些操作。为训练此类统一行为，我们提出了一种三阶段渐进式强化学习策略，并设计了逐步GRPO以应对由记忆操作引发的稀疏且不连续的奖励问题。在五个长程基准测试上的实验表明，AgeMem在多种LLM骨干模型上均持续优于强记忆增强基线方法，实现了更高的任务性能、更高质量的长期记忆以及更高效的上下文使用。