Long-horizon agents rely on memory mechanisms to compress interaction history, but optimizing memory writing faces a distinct credit assignment challenge: a memory update may be rewarded or penalized due to downstream tool failures, noisy observations, or reasoning errors rather than its own contribution. This causally entangled credit can lead agents to discard useful evidence or preserve irrelevant information. We propose HiMPO, a Hindsight-Informed Memory Policy Optimization framework for assigning less-entangled credit to memory-writing actions in long-horizon agents. HiMPO first estimates the local utility of a memory update by comparing the task-relevant information recoverable from the previous and updated memories under the same pre-write state. It then uses hindsight relevance as a bounded retrospective filter that attenuates memory credit when local utility is not supported by the target outcome. The resulting memory-specific advantage is applied only to memory tokens, while trajectory-level rewards optimize the rest of the agent behavior. Across judge-based open-domain tasks and objective compressive-memory QA, HiMPO improves over strong memory-based and RL-based baselines while preserving compressed-context efficiency. Controlled interventions further show that HiMPO reduces blame leakage from tool-induced errors and improves attribution fidelity of memory updates.

翻译：长周期智能体依赖记忆机制压缩交互历史，但优化记忆写入面临独特的信用分配挑战：某次记忆更新可能因下游工具故障、噪声观测或推理错误而获得奖励或惩罚，而非源于其自身贡献。这种因果纠缠的信用会导致智能体丢弃有效证据或保留无关信息。我们提出HiMPO——一种后见知情记忆策略优化框架，旨在为长周期智能体的记忆写入动作分配更少纠缠的信用。HiMPO首先通过比较在相同写入前状态下，从先前记忆与更新记忆中可恢复的任务相关信息，来估计单次记忆更新的局部效用；随后将后见相关性作为有界回溯滤波器，当局部效用未得到目标结果支持时，对记忆信用进行衰减。由此生成的记忆专属优势仅作用于记忆令牌，而轨迹层级奖励则优化智能体的其余行为。在基于评判的开放域任务与客观压缩记忆问答任务中，HiMPO在保持压缩上下文效率的同时，超越了基于强记忆与强强化学习的基线方法。受控干预实验进一步表明，HiMPO能减少由工具诱发错误导致的归责泄漏，并提升记忆更新的归因保真度。