Accurate and robust trajectory predictions of road users are needed to enable safe automated driving. To do this, machine learning models are often used, which can show erratic behavior when presented with previously unseen inputs. In this work, two environment-aware models (MotionCNN and MultiPath++) and two common baselines (Constant Velocity and an LSTM) are benchmarked for robustness against various perturbations that simulate functional insufficiencies observed during model deployment in a vehicle: unavailability of road information, late detections, and noise. Results show significant performance degradation under the presence of these perturbations, with errors increasing up to +1444.8\% in commonly used trajectory prediction evaluation metrics. Training the models with similar perturbations effectively reduces performance degradation, with error increases of up to +87.5\%. We argue that despite being an effective mitigation strategy, data augmentation through perturbations during training does not guarantee robustness towards unforeseen perturbations, since identification of all possible on-road complications is unfeasible. Furthermore, degrading the inputs sometimes leads to more accurate predictions, suggesting that the models are unable to learn the true relationships between the different elements in the data.
翻译:准确且鲁棒的道路使用者轨迹预测是实现安全自动驾驶的必要条件。为此,常采用机器学习模型,但这些模型在面对未见过的输入时可能表现出异常行为。本研究对两种环境感知模型(MotionCNN和MultiPath++)以及两种常见基线模型(恒定速度模型和LSTM)进行鲁棒性基准测试,测试对象为模拟车辆部署中观察到的功能缺陷的各种扰动,包括道路信息不可用、检测延迟和噪声干扰。结果表明,在这些扰动下模型性能显著下降,在常用轨迹预测评估指标中误差增幅高达+1444.8%。通过使用类似扰动训练模型可有效减轻性能退化,误差增幅降至+87.5%。我们认为尽管数据增强通过训练引入扰动是有效的缓解策略,但无法保证对未知扰动的鲁棒性,因为识别所有可能的道路突发情况是不可行的。此外,输入退化有时反而会提高预测精度,表明模型未能学习数据中各要素间的真实关联。