Effective memory management is essential for large language model agents to navigate long-horizon tasks. Recent research has explored using Reinforcement Learning to develop specialized memory manager agents. However, existing approaches rely on final task performance as the primary reward, which results in severe reward sparsity and ineffective credit assignment, providing insufficient guidance for individual memory operations. To this end, we propose Fine-Mem, a unified framework designed for fine-grained feedback alignment. First, we introduce a Chunk-level Step Reward to provide immediate step-level supervision via auxiliary chunk-specific question answering tasks. Second, we devise Evidence-Anchored Reward Attribution to redistribute global rewards by anchoring credit to key memory operations, based on the specific memory items utilized as evidence in reasoning. Together, these components enable stable policy optimization and align local memory operations with the long-term utility of memory. Experiments on Memalpha and MemoryAgentBench demonstrate that Fine-Mem consistently outperforms strong baselines, achieving superior success rates across various sub-tasks. Further analysis reveals its adaptability and strong generalization capabilities across diverse model configurations and backbones.

翻译：有效的记忆管理对于大型语言模型智能体处理长程任务至关重要。近期研究探索了利用强化学习训练专用记忆管理智能体的方法。然而，现有方法主要依赖最终任务性能作为核心奖励信号，这导致严重的奖励稀疏性和低效的信用分配，无法为具体记忆操作提供充分指导。为此，我们提出Fine-Mem——一个为精细化反馈对齐设计的统一框架。首先，我们引入分块级步进奖励机制，通过辅助性的分块特定问答任务提供即时步进级监督。其次，我们设计了证据锚定式奖励归因方法，依据推理过程中作为证据使用的具体记忆条目，将全局奖励重新分配并锚定至关键记忆操作。这两个组件共同实现了稳定的策略优化，并使局部记忆操作与记忆的长期效用保持一致。在Memalpha和MemoryAgentBench基准上的实验表明，Fine-Mem始终优于现有强基线模型，在各类子任务中均取得了更高的成功率。进一步分析揭示了该框架在不同模型配置与骨干网络间具有良好的适应性与强大的泛化能力。