Autonomous Vehicles (AVs), especially vision-based AVs, are rapidly being deployed without human operators. As AVs operate in safety-critical environments, understanding their robustness in an adversarial environment is an important research problem. Prior physical adversarial attacks on vision-based autonomous vehicles predominantly target immediate safety failures (e.g., a crash, a traffic-rule violation, or a transient lane departure) by inducing a short-lived perception or control error. This paper shows a qualitatively different risk: a long-horizon route integrity compromise, where an attacker gradually steers a victim AV away from its intended route and into an attacker-chosen destination while the victim continues to drive ``normally.'' This will not pose a danger to the victim vehicle itself, but also to potential passengers sitting inside the vehicle, who may not notice the route changes. In this paper, we design and implement the first adversarial framework, called JackZebra, which performs route-level hijacking of a vision-based end-to-end driving stack using a physically plausible attacker vehicle with a reconfigurable display and a camera sensor mounted on the rear. The central challenge is temporal persistence: adversarial influence must remain effective in changing viewpoints, lighting, weather, traffic, and the victim's continual replanning -- without triggering conspicuous failures. Our key insight is to treat route hijacking as a closed-loop control problem and to convert adversarial patches into steering primitives that can be selected online via an interactive adjustment loop based on observed victim behavior using the rear camera. Our evaluations in both simulated and real-world scenarios show that JackZebra can successfully hijack victim vehicles to deviate from original routes and stop at places designated by the adversary with a high success rate.

翻译：自动驾驶汽车，尤其是基于视觉的自动驾驶汽车，正迅速部署于无人类操作员的环境。由于自动驾驶汽车在安全关键场景中运行，理解其在对抗性环境中的鲁棒性是一项重要的研究课题。先前针对基于视觉的自动驾驶汽车的物理对抗攻击，主要通过在短时间内诱发感知或控制错误，直接造成即时安全故障（例如碰撞、违反交通规则或短暂的车道偏离）。本文揭示了一种性质不同的风险：一种长时程的路线完整性破坏。在此攻击中，攻击者逐步引导受害自动驾驶汽车偏离其预定路线，驶向攻击者指定的目的地，而受害车辆在表面上仍“正常”行驶。这不仅对受害车辆本身构成潜在危险，也对车内可能未察觉路线变更的乘客构成威胁。本文设计并实现了首个对抗性框架——JackZebra，该框架利用一辆物理上可行的攻击车辆（配备可重构显示器及安装于车尾的摄像头传感器），对基于视觉的端到端驾驶系统实施路线级劫持。其核心挑战在于时间持续性：对抗性影响必须在变化的视角、光照、天气、交通条件以及受害车辆持续的重新规划中保持有效，同时避免触发明显的故障。我们的关键洞见在于将路线劫持视为一个闭环控制问题，并将对抗性贴片转化为转向原语，这些原语可通过基于后置摄像头观察到的受害车辆行为的交互式调整循环在线选择。我们在仿真和真实场景中的评估表明，JackZebra能够以高成功率成功劫持受害车辆，使其偏离原始路线，并停在攻击者预设的位置。