Self-evolving LLM agents update their internal state across sessions, often by writing and reusing long-term memory. This design improves performance on long-horizon tasks but creates a security risk: untrusted external content observed during a benign session can be stored as memory and later treated as instruction. We study this risk and formalize a persistent attack we call a Zombie Agent, where an attacker covertly implants a payload that survives across sessions, effectively turning the agent into a puppet of the attacker. We present a black-box attack framework that uses only indirect exposure through attacker-controlled web content. The attack has two phases. During infection, the agent reads a poisoned source while completing a benign task and writes the payload into long-term memory through its normal update process. During trigger, the payload is retrieved or carried forward and causes unauthorized tool behavior. We design mechanism-specific persistence strategies for common memory implementations, including sliding-window and retrieval-augmented memory, to resist truncation and relevance filtering. We evaluate the attack on representative agent setups and tasks, measuring both persistence over time and the ability to induce unauthorized actions while preserving benign task quality. Our results show that memory evolution can convert one-time indirect injection into persistent compromise, which suggests that defenses focused only on per-session prompt filtering are not sufficient for self-evolving agents.
翻译:自我进化的LLM代理通过跨会话更新内部状态(通常借助长期记忆的写入与复用)来提升长期任务性能,但这种设计也带来了安全风险:良性会话期间观察到的不可信外部内容可能被存储为记忆,并在后续被当作指令执行。本研究系统分析了该风险,并形式化定义了一种我们称为"僵尸代理"的持久性攻击——攻击者可隐蔽植入跨会话存活的恶意载荷,从而将代理转化为攻击者的傀儡。我们提出一种仅通过攻击者控制的网页内容进行间接暴露的黑盒攻击框架。该攻击包含两个阶段:感染阶段,代理在执行良性任务时读取被污染的信息源,并通过正常更新流程将恶意载荷写入长期记忆;触发阶段,恶意载荷被检索或传递,进而引发未授权的工具行为。针对滑动窗口记忆和检索增强记忆等常见记忆实现机制,我们设计了抗截断与相关性过滤的机制特异性持久化策略。通过在典型代理配置和任务场景中的评估,我们同时测量了攻击的跨会话持久性和诱导未授权行为的能力(同时保持良性任务质量)。实验结果表明,记忆进化机制可将单次间接注入转化为持久性入侵,这意味着仅关注单会话提示过滤的防御策略对自我进化代理而言是不充分的。