Autonomous vehicles ought to predict the surrounding agents' trajectories to allow safe maneuvers in uncertain and complex traffic situations. As companies increasingly apply trajectory prediction in the real world, security becomes a relevant concern. In this paper, we focus on backdoors - a security threat acknowledged in other fields but so far overlooked for trajectory prediction. To this end, we describe and investigate four triggers that could affect trajectory prediction. We then show that these triggers (for example, a braking vehicle), when correlated with a desired output (for example, a curve) during training, cause the desired output of a state-of-the-art trajectory prediction model. In other words, the model has good benign performance but is vulnerable to backdoors. This is the case even if the trigger maneuver is performed by a non-casual agent behind the target vehicle. As a side-effect, our analysis reveals interesting limitations within trajectory prediction models. Finally, we evaluate a range of defenses against backdoors. While some, like simple offroad checks, do not enable detection for all triggers, clustering is a promising candidate to support manual inspection to find backdoors.
翻译:自动驾驶汽车需要预测周围智能体的轨迹,以便在不确定且复杂的交通场景中实现安全操控。随着各行业在现实世界中越来越多地应用轨迹预测,安全性已成为一个重要关注点。本文聚焦于后门攻击——这一安全威胁在其他领域已获公认,但在轨迹预测中至今仍被忽视。为此,我们描述并研究了四种可能影响轨迹预测的触发器。进而证明,当这些触发器(例如制动车辆)在训练过程中与预期输出(例如弯道)相关联时,会导致先进轨迹预测模型产生预期输出。换言之,模型在正常状况下表现良好,却易受后门攻击影响。即使触发操作由目标车辆后方非因果智能体执行,这一现象依然存在。作为副产品,我们的分析揭示了轨迹预测模型中存在的有趣局限性。最后,我们评估了一系列针对后门攻击的防御措施。虽然部分方法(如简单的越野检测)无法对所有触发器实现有效检测,但聚类方法作为支持人工检查以发现后门攻击的候选方案具有前景。