Accurate and robust trajectory prediction of neighboring agents is critical for autonomous vehicles traversing in complex scenes. Most methods proposed in recent years are deep learning-based due to their strength in encoding complex interactions. However, unplausible predictions are often generated since they rely heavily on past observations and cannot effectively capture the transient and contingency interactions from sparse samples. In this paper, we propose a hierarchical hybrid framework of deep learning (DL) and reinforcement learning (RL) for multi-agent trajectory prediction, to cope with the challenge of predicting motions shaped by multi-scale interactions. In the DL stage, the traffic scene is divided into multiple intermediate-scale heterogenous graphs based on which Transformer-style GNNs are adopted to encode heterogenous interactions at intermediate and global levels. In the RL stage, we divide the traffic scene into local sub-scenes utilizing the key future points predicted in the DL stage. To emulate the motion planning procedure so as to produce trajectory predictions, a Transformer-based Proximal Policy Optimization (PPO) incorporated with a vehicle kinematics model is devised to plan motions under the dominant influence of microscopic interactions. A multi-objective reward is designed to balance between agent-centric accuracy and scene-wise compatibility. Experimental results show that our proposal matches the state-of-the-arts on the Argoverse forecasting benchmark. It's also revealed by the visualized results that the hierarchical learning framework captures the multi-scale interactions and improves the feasibility and compliance of the predicted trajectories.

翻译：准确且鲁棒地预测周围智能体的轨迹对于自动驾驶车辆在复杂场景中的行驶至关重要。近年来提出的多数方法基于深度学习，因其在编码复杂交互方面的优势。然而，由于这些方法过度依赖历史观测数据，且无法从稀疏样本中有效捕捉瞬态与偶发性交互，常常产生不合理的预测。本文提出一种融合深度学习与强化学习的层次化混合框架，用于多智能体轨迹预测，以应对多尺度交互塑造的运动预测挑战。在深度学习阶段，将交通场景划分为多个中等尺度异构图，并基于Transformer型图神经网络在中等及全局层次编码异构交互。在强化学习阶段，利用深度学习阶段预测的关键未来点将交通场景划分为局部子场景。为模拟运动规划过程以生成轨迹预测，本文设计了一种融合车辆运动学模型的Transformer型近端策略优化算法，在微观交互主导影响下规划运动。同时设计多目标奖励函数以平衡智能体中心精度与场景兼容性。实验结果表明，所提方法在Argoverse预测基准上达到当前最优性能。可视化结果进一步揭示，该层次化学习框架能够捕获多尺度交互，并提升预测轨迹的可行性与合规性。