Embodied Artificial Intelligence (AI) promises to handle edge cases in robotic vehicle systems where data is scarce by using common-sense reasoning grounded in perception and action to generalize beyond training distributions and adapt to novel real-world situations. These capabilities, however, also create new security risks. In this paper, we introduce CHAI (Command Hijacking against embodied AI), a physical environment indirect prompt injection attack that exploits the multimodal language interpretation abilities of AI models. CHAI embeds deceptive natural language instructions, such as misleading signs, in visual input, systematically searches the token space, builds a dictionary of prompts, and guides an attacker model to generate Visual Attack Prompts. We evaluate CHAI on four LVLM agents: drone emergency landing, autonomous driving, aerial object tracking, and on a real robotic vehicle. Our experiments show that CHAI consistently outperforms state-of-the-art attacks. By exploiting the semantic and multimodal reasoning strengths of next-generation embodied AI systems, CHAI underscores the urgent need for defenses that extend beyond traditional adversarial robustness.

翻译：具身人工智能（AI）有望通过利用基于感知与行动的常识推理来泛化超出训练分布的范围，并适应新颖的现实世界场景，从而解决机器人车辆系统中数据稀缺的边缘案例。然而，这些能力也带来了新的安全风险。本文提出了CHAI（针对具身AI的命令劫持攻击），这是一种物理环境间接提示注入攻击，它利用了AI模型的多模态语言理解能力。CHAI在视觉输入中嵌入欺骗性的自然语言指令（例如误导性标志），系统性地搜索词元空间，构建提示词典，并引导攻击者模型生成视觉攻击提示。我们在四个LVLM智能体上评估了CHAI：无人机紧急降落、自动驾驶、空中目标跟踪，以及在一个真实的机器人车辆上。实验结果表明，CHAI在性能上持续优于最先进的攻击方法。通过利用下一代具身AI系统的语义与多模态推理优势，CHAI凸显了超越传统对抗鲁棒性的防御措施的迫切需求。