Multimodal Large Language Models (MLLMs) are increasingly deployed in stateless systems, such as autonomous driving and robotics. This paper investigates a novel threat: Semantic-Aware Hijacking. We explore the feasibility of hijacking multiple stateless decisions simultaneously using a single universal perturbation. We introduce the Semantic-Aware Universal Perturbation (SAUP), which acts as a semantic router, "actively" perceiving input semantics and routing them to distinct, attacker-defined targets. To achieve this, we conduct theoretical and empirical analysis on the geometric properties in the latent space. Guided by these insights, we propose the Semantic-Oriented (SORT) optimization strategy and annotate a new dataset with fine-grained semantics to evaluate performance. Extensive experiments on three representative MLLMs demonstrate the fundamental feasibility of this attack, achieving a 66% attack success rate over five targets using a single frame against Qwen.
翻译:多模态大语言模型(MLLMs)正日益部署于自动驾驶和机器人等无状态系统中。本文研究了一种新型威胁:语义感知劫持。我们探讨了使用单一通用扰动同时劫持多个无状态决策的可行性。我们引入了语义感知通用扰动(SAUP),它充当语义路由器,“主动”感知输入语义并将其路由至攻击者定义的不同目标。为实现此目标,我们对潜在空间中的几何特性进行了理论与实证分析。基于这些洞见,我们提出了语义导向(SORT)优化策略,并标注了一个具有细粒度语义的新数据集以评估性能。在三个代表性MLLMs上的大量实验证明了该攻击的根本可行性:针对Qwen模型,使用单帧图像对五个目标实现了66%的攻击成功率。