Motion forecasting is crucial in enabling autonomous vehicles to anticipate the future trajectories of surrounding agents. To do so, it requires solving mapping, detection, tracking, and then forecasting problems, in a multi-step pipeline. In this complex system, advances in conventional forecasting methods have been made using curated data, i.e., with the assumption of perfect maps, detection, and tracking. This paradigm, however, ignores any errors from upstream modules. Meanwhile, an emerging end-to-end paradigm, that tightly integrates the perception and forecasting architectures into joint training, promises to solve this issue. So far, however, the evaluation protocols between the two methods were incompatible and their comparison was not possible. In fact, and perhaps surprisingly, conventional forecasting methods are usually not trained nor tested in real-world pipelines (e.g., with upstream detection, tracking, and mapping modules). In this work, we aim to bring forecasting models closer to real-world deployment. First, we propose a unified evaluation pipeline for forecasting methods with real-world perception inputs, allowing us to compare the performance of conventional and end-to-end methods for the first time. Second, our in-depth study uncovers a substantial performance gap when transitioning from curated to perception-based data. In particular, we show that this gap (1) stems not only from differences in precision but also from the nature of imperfect inputs provided by perception modules, and that (2) is not trivially reduced by simply finetuning on perception outputs. Based on extensive experiments, we provide recommendations for critical areas that require improvement and guidance towards more robust motion forecasting in the real world. We will release an evaluation library to benchmark models under standardized and practical conditions.

翻译：运动预测是实现自动驾驶车辆预测周围智能体未来轨迹的核心技术。它需通过多阶段流水线解决地图构建、目标检测、目标跟踪及预测问题。在该复杂系统中，传统预测方法的进展通常基于人工精炼数据（即假设完美地图、检测和跟踪结果），但这种范式忽视了上游模块的误差。与此同时，新兴的端到端范式通过将感知与预测架构紧密集成并进行联合训练，有望解决这一问题。然而，目前两种方法的评估协议互不兼容，尚无法直接比较。令人意外的是，传统预测方法通常未在真实流水线（如包含上游检测、跟踪和地图构建模块）中进行训练或测试。本研究旨在推动预测模型向真实部署场景靠拢。首先，我们提出基于真实感知输入的统一评估流水线，首次实现了传统方法与端到端方法的性能对比。其次，深度研究发现：从精炼数据转向感知数据后，性能出现显著下降。具体而言，该性能差距（1）不仅源于精度差异，更源于感知模块提供的不完美输入的本质特性，（2）且无法通过简单微调感知输出而显著缩小。基于大量实验，我们针对需改进的关键领域提出建议，并为更鲁棒的真实世界运动预测提供指导。我们将发布标准化实用条件下的模型基准评估库。