Indirect prompt injection poses a critical threat to LLM agents that interact with untrusted external data, yet automated attack methods--proven effective for jailbreaking--remain underexplored in realistic agentic settings. We present a comprehensive empirical evaluation of automated prompt injection attacks against LLM agents, adapting both white-box (GCG) and black-box (TAP) methods to the agentic setting within the AgentDojo framework. We evaluate across 80 task pairs spanning four domains and multiple models, and find that black-box optimization substantially outperforms gradient-based methods, a gap we attribute to GCG's optimization instability under reasonable compute budgets. We also find that TAP's effectiveness depends on the attacker model, as both general capability and safety tuning affect attack success--stronger models produce more effective injections, while safety-tuned attackers can refuse to generate adversarial prompts. Task-universal attacks transfer effectively to unseen tasks and out-of-distribution domains, but attacks optimized on smaller open-source models do not transfer to frontier models like GPT-5. These findings highlight automated prompt injection as a credible but model-dependent threat, with significant barriers remaining for model-agnostic exploitation.

翻译：间接提示注入对与不可信外部数据交互的大型语言模型（LLM）智能体构成严重威胁，然而，在现实智能体场景中，经证明对越狱有效的自动化攻击方法仍未得到充分探索。我们提出了针对LLM智能体的自动化提示注入攻击的全面实证评估，将白盒（GCG）和黑盒（TAP）方法适配到AgentDojo框架中的智能体设置。我们在跨越四个领域和多个模型的80个任务对上进行评估，发现黑盒优化显著优于基于梯度的方法，我们将这一差距归因于GCG在合理计算预算下的优化不稳定性。我们还发现，TAP的有效性取决于攻击者模型，因为通用能力和安全调参都会影响攻击成功率——更强的模型产生更有效的注入，而经过安全调参的攻击者可能拒绝生成对抗性提示。任务通用攻击能够有效迁移至未见任务和分布外领域，但在较小开源模型上优化的攻击无法迁移至GPT-5等前沿模型。这些发现表明，自动提示注入是一种可信但依赖于模型的威胁，在实现模型无关的利用方面仍存在重大障碍。