Robotic manipulation policies are increasingly empowered by \textit{large language models} (LLMs) and \textit{vision-language models} (VLMs), leveraging their understanding and perception capabilities. Recently, inference-time attacks against robotic manipulation have been extensively studied, yet backdoor attacks targeting model supply chain security in robotic policies remain largely unexplored. To fill this gap, we propose \texttt{TrojanRobot}, a backdoor injection framework for model supply chain attack scenarios, which embeds a malicious module into modular robotic policies via backdoor relationships to manipulate the LLM-to-VLM pathway and compromise the system. Our vanilla design instantiates this module as a backdoor-finetuned VLM. To further enhance attack performance, we propose a prime scheme by introducing the concept of \textit{LVLM-as-a-backdoor}, which leverages \textit{in-context instruction learning} (ICIL) to steer \textit{large vision-language model} (LVLM) behavior through backdoored system prompts. Moreover, we develop three types of prime attacks, \textit{permutation}, \textit{stagnation}, and \textit{intentional}, achieving flexible backdoor attack effects. Extensive physical-world and simulator experiments on 18 real-world manipulation tasks and 4 VLMs verify the superiority of proposed \texttt{TrojanRobot}
翻译:机器人操作策略正日益依赖大型语言模型(LLMs)和视觉语言模型(VLMs)的理解与感知能力。尽管面向机器人操作的推理时攻击已被广泛研究,但针对机器人策略中模型供应链安全的后门攻击仍鲜有探讨。为填补这一空白,我们提出TrojanRobot——一个面向模型供应链攻击场景的后门注入框架,该框架通过后门关系将恶意模块嵌入模块化机器人策略中,以操纵LLM-to-VLM通路并破坏系统。我们的基准设计将该模块实现为一个经后门微调的VLM。为进一步提升攻击性能,我们提出一种进阶方案,引入“将LVLM作为后门”的概念,利用上下文指令学习(ICIL)通过带有后门的系统提示引导大型视觉语言模型(LVLM)行为。此外,我们开发了三种进阶攻击类型:置换、停滞和故意攻击,实现了灵活的后门攻击效果。在18项真实操作任务和4种VLM上进行的广泛物理世界与仿真实验验证了所提出的TrojanRobot的优越性。