Interactive LLM agents are becoming part of daily work, but they do not reliably become easier to work with over time: a correction remembered in one session may still be violated in the next. We study this gap between preference access and preference compliance. In tasks derived from anonymized real-user friction cases, Mem0 memory still leaves 57.5% of applicable preference checks violated. We introduce Test-time Rule Acquisition and Compiled Enforcement (TRACE), a drop-in skill-layer pipeline for coding-agent runtimes that mines user corrections, rewrites them as atomic rules, and compiles them into runtime checks that must pass before an agent completes future tasks. Unlike runtime checks written ahead of time by developers, TRACE skills come from the user's own chat corrections. We evaluate TRACE with simulated user-in-the-loop experiments on ClawArena coding-agent tasks and MemoryArena-derived memory-intensive tasks. On ClawArena, TRACE reduces held-out preference violation from 100.0% to 37.6% on in-distribution tasks and from 100.0% to 2.0% on out-of-distribution tasks. On MemoryArena-derived tasks, TRACE reduces in-distribution violation from 100.0% to 60.5% while matching or exceeding the strongest memory baseline on task pass. These results suggest that compiling corrections into runtime enforcement can address a repeated-friction failure mode that memory alone does not reliably solve, reducing the need for users to restate the same correction across future sessions. Experiment code is available at https://github.com/YujunZhou/TRACE_exp, and the deployable skill is available at https://github.com/YujunZhou/tellonce.
翻译:交互式大语言模型(LLM)Agent正日益融入日常工作,但它们并不会随着时间推移而变得更加可靠:某个会话中记住的修正,可能在下一个会话中仍被违反。我们研究了偏好访问与偏好遵从之间的这一差距。在基于匿名真实用户摩擦案例的任务中,Mem0记忆系统仍导致57.5%的相关偏好检查被违反。我们提出测试时规则获取与编译强制(TRACE),这是一种即插即用的技能层流水线,用于编码Agent的运行时:该流水线挖掘用户修正,将其重写为原子规则,并编译为运行时检查——这些检查必须在Agent完成后续任务前通过。与开发者预先编写的运行时检查不同,TRACE技能源自用户自身的聊天修正。我们在ClawArena编码Agent任务和MemoryArena衍生的内存密集型任务上,通过模拟用户参与的实验评估了TRACE。在ClawArena上,TRACE将分布内任务的留出偏好违反率从100.0%降至37.6%,将分布外任务的留出偏好违反率从100.0%降至2.0%。在MemoryArena衍生任务上,TRACE将分布内任务违反率从100.0%降至60.5%,同时在任务通过率上达到或超过最强的记忆基线。这些结果表明,将修正编译为运行时强制机制可以解决记忆单独无法可靠解决的重复摩擦失败模式,从而减少用户在后续会话中重复表述相同修正的需求。实验代码见https://github.com/YujunZhou/TRACE_exp,可部署技能见https://github.com/YujunZhou/tellonce。