Accurate motion forecasting is critical for autonomous driving, yet most predictors rely on multi-object tracking (MOT) with identity association, assuming that objects are correctly and continuously tracked. When tracking fails due to, e.g., occlusion, identity switches, or missed detections, prediction quality degrades and safety risks increase. We present \textbf{HiMAP}, a tracking-free, trajectory prediction framework that remains reliable under MOT failures. HiMAP converts past detections into spatiotemporally invariant historical occupancy maps and introduces a historical query module that conditions on the current agent state to iteratively retrieve agent-specific history from unlabeled occupancy representations. The retrieved history is summarized by a temporal map embedding and, together with the final query and map context, drives a DETR-style decoder to produce multi-modal future trajectories. This design lifts identity reliance, supports streaming inference via reusable encodings, and serves as a robust fallback when tracking is unavailable. On Argoverse~2, HiMAP achieves performance comparable to tracking-based methods while operating without IDs, and it substantially outperforms strong baselines in the no-tracking setting, yielding relative gains of 11\% in FDE, 12\% in ADE, and a 4\% reduction in MR over a fine-tuned QCNet. Beyond aggregate metrics, HiMAP delivers stable forecasts for all agents simultaneously without waiting for tracking to recover, highlighting its practical value for safety-critical autonomy. The code is available under: https://github.com/XuYiMing83/HiMAP.

翻译：精确的运动预测对于自动驾驶至关重要，然而大多数预测器依赖于带身份关联的多目标跟踪（MOT），并假设目标被正确且连续地跟踪。当跟踪因遮挡、身份切换或漏检等原因失败时，预测质量会下降，安全风险随之增加。本文提出 \textbf{HiMAP}，一种无需跟踪的轨迹预测框架，在 MOT 失效时仍能保持可靠性。HiMAP 将过去的检测结果转换为时空不变的历史占用地图，并引入一个历史查询模块，该模块以当前智能体状态为条件，从未标记的占用表示中迭代检索特定于智能体的历史信息。检索到的历史通过时序地图嵌入进行汇总，并与最终查询及地图上下文共同驱动一个 DETR 风格的解码器，以生成多模态的未来轨迹。这一设计消除了对身份标识的依赖，通过可重用的编码支持流式推理，并在跟踪不可用时作为鲁棒的回退方案。在 Argoverse~2 数据集上，HiMAP 在不使用身份标识的情况下取得了与基于跟踪的方法相当的性能，并在无跟踪设置中显著优于强基线模型，相对于微调后的 QCNet，在最终位移误差（FDE）上相对提升 11\%，平均位移误差（ADE）上提升 12\%，并降低了 4\% 的漏检率（MR）。除综合指标外，HiMAP 能够同时为所有智能体提供稳定的预测，无需等待跟踪恢复，突显了其在安全关键自动驾驶系统中的实用价值。代码发布于：https://github.com/XuYiMing83/HiMAP。