AI agents that autonomously interact with external tools and environments show great promise across real-world applications. However, the external data which agent consumes also leads to the risk of indirect prompt injection attacks, where malicious instructions embedded in third-party content hijack agent behavior. Guided by benchmarks, such as AgentDojo, there has been significant amount of progress in developing defense against the said attacks. As the technology continues to mature, and that agents are increasingly being relied upon for more complex tasks, there is increasing pressing need to also evolve the benchmark to reflect threat landscape faced by emerging agentic systems. In this work, we reveal three fundamental flaws in current benchmarks and push the frontier along these dimensions: (i) lack of dynamic open-ended tasks, (ii) lack of helpful instructions, and (iii) simplistic user tasks. To bridge this gap, we introduce AgentDyn, a manually designed benchmark featuring 60 challenging open-ended tasks and 560 injection test cases across Shopping, GitHub, and Daily Life. Unlike prior static benchmarks, AgentDyn requires dynamic planning and incorporates helpful third-party instructions. Our evaluation of ten state-of-the-art defenses suggests that almost all existing defenses are either not secure enough or suffer from significant over-defense, revealing that existing defenses are still far from real-world deployment. Our benchmark is available at https://github.com/leolee99/AgentDyn.

翻译：能够自主与外部工具和环境交互的AI智能体在现实世界应用中展现出巨大潜力。然而，智能体所处理的外部数据也带来了间接提示注入攻击的风险，即嵌入在第三方内容中的恶意指令可能劫持智能体行为。在诸如AgentDojo等基准的指导下，针对此类攻击的防御研究已取得显著进展。随着技术持续成熟，且智能体日益被依赖以执行更复杂的任务，也迫切需要同步演进基准，以反映新兴智能体系统所面临的威胁态势。本工作中，我们揭示了当前基准存在的三个根本缺陷，并沿着这些维度推进前沿：(i) 缺乏动态开放式任务，(ii) 缺乏有益指令，以及 (iii) 用户任务过于简单。为弥补这一差距，我们提出了AgentDyn，这是一个人工设计的基准，包含60个具有挑战性的开放式任务以及横跨购物、GitHub和日常生活三大领域的560个注入测试用例。与先前的静态基准不同，AgentDyn要求动态规划，并融入了有益的第三方指令。我们对十种最先进防御方法的评估表明，几乎所有现有防御要么安全性不足，要么存在严重的过度防御问题，这揭示出现有防御距离实际部署仍有很大差距。我们的基准已在 https://github.com/leolee99/AgentDyn 发布。