Control-flow hijacking attacks manipulate orchestration mechanisms in multi-agent systems into performing unsafe actions that compromise the system and exfiltrate sensitive information. Recently proposed defenses, such as LlamaFirewall, rely on alignment checks of inter-agent communications to ensure that all agent invocations are "related to" and "likely to further" the original objective. We start by demonstrating control-flow hijacking attacks that evade these defenses even if alignment checks are performed by advanced LLMs. We argue that the safety and functionality objectives of multi-agent systems fundamentally conflict with each other. This conflict is exacerbated by the brittle definitions of "alignment" and the checkers' incomplete visibility into the execution context. We then propose, implement, and evaluate ControlValve, a new defense inspired by the principles of control-flow integrity and least privilege. ControlValve (1) generates permitted control-flow graphs for multi-agent systems, and (2) enforces that all executions comply with these graphs, along with contextual rules (generated in a zero-shot manner) for each agent invocation.

翻译：控制流劫持攻击通过操纵多智能体系统中的协同机制，诱使系统执行危害系统安全并泄露敏感信息的危险操作。近期提出的防御方案（如LlamaFirewall）依赖于对智能体间通信的对齐校验，以确保所有智能体调用均"与原始目标相关"且"可能推进目标实现"。本文首先展示了即使采用先进大语言模型进行对齐校验，仍能规避现有防御的控制流劫持攻击。我们认为多智能体系统的安全目标与功能目标存在根本性冲突，这种冲突因"对齐"定义的脆弱性及检查器对执行上下文可见性不足而加剧。随后，我们基于控制流完整性与最小权限原则，提出、实现并评估了新型防御方案ControlValve。该方案具备两大核心功能：(1) 为多智能体系统生成许可控制流图；(2) 强制所有执行过程遵循该控制流图，并为每次智能体调用生成零样本上下文规则。