Autonomous vehicles ought to predict the surrounding agents' trajectories to allow safe maneuvers in uncertain and complex traffic situations. As companies increasingly apply trajectory prediction in the real world, security becomes a relevant concern. In this paper, we focus on backdoors - a security threat acknowledged in other fields but so far overlooked for trajectory prediction. To this end, we describe and investigate four triggers that could affect trajectory prediction. We then show that these triggers (for example, a braking vehicle), when correlated with a desired output (for example, a curve) during training, cause the desired output of a state-of-the-art trajectory prediction model. In other words, the model has good benign performance but is vulnerable to backdoors. This is the case even if the trigger maneuver is performed by a non-casual agent behind the target vehicle. As a side-effect, our analysis reveals interesting limitations within trajectory prediction models. Finally, we evaluate a range of defenses against backdoors. While some, like simple offroad checks, do not enable detection for all triggers, clustering is a promising candidate to support manual inspection to find backdoors.
翻译:自动驾驶汽车应当预测周围智能体的轨迹,以便在不确定且复杂的交通环境中实现安全操控。随着企业在实际场景中越来越多地应用轨迹预测技术,安全性问题成为重要关切。本文聚焦后门攻击——这一在其他领域已被认知的安全威胁,在轨迹预测领域迄今未得到充分研究。为此,我们描述并研究了四种可能影响轨迹预测的触发器,进而证明:当这些触发器(例如制动车辆)在训练过程中与预期输出(例如弯道)相关联时,会导致最先进的轨迹预测模型输出预期结果。换言之,模型在正常场景下表现良好,但面临后门攻击的脆弱性——即使触发动作是由目标车辆后方非因果相关的智能体执行的。作为附带发现,我们的分析揭示了轨迹预测模型的有趣局限性。最后,我们评估了一系列针对后门攻击的防御措施。尽管部分方法(如简单越野检测)无法在全部触发器场景下实现检测,聚类方法仍是辅助人工检查以发现后门攻击的有前景方案。