Agentic AI systems powered by large language models (LLMs) and endowed with planning, tool use, memory, and autonomy, are emerging as powerful, flexible platforms for automation. Their ability to autonomously execute tasks across web, software, and physical environments creates new and amplified security risks, distinct from both traditional AI safety and conventional software security. This survey outlines a taxonomy of threats specific to agentic AI, reviews recent benchmarks and evaluation methodologies, and discusses defense strategies from both technical and governance perspectives. We synthesize current research and highlight open challenges, aiming to support the development of secure-by-design agent systems.
翻译:由大语言模型(LLMs)驱动,并具备规划、工具使用、记忆和自主能力的智能体AI系统,正作为强大而灵活的自動化平台迅速兴起。它们能够在网络、软件和物理环境中自主执行任务,这产生了新的、加剧的安全风险,这些风险既不同于传统的AI安全,也不同于常规的软件安全。本综述概述了针对智能体AI特有威胁的分类法,回顾了最近的基准测试和评估方法,并从技术和治理两个角度讨论了防御策略。我们综合了当前的研究并强调了开放性的挑战,旨在支持设计即安全的智能体系统的开发。