Trajectory prediction in autonomous driving relies on accurate representation of all relevant contexts of the driving scene, including traffic participants, road topology, traffic signs, as well as their semantic relations to each other. Despite increased attention to this issue, most approaches in trajectory prediction do not consider all of these factors sufficiently. We present SemanticFormer, an approach for predicting multimodal trajectories by reasoning over a semantic traffic scene graph using a hybrid approach. It utilizes high-level information in the form of meta-paths, i.e. trajectories on which an agent is allowed to drive from a knowledge graph which is then processed by a novel pipeline based on multiple attention mechanisms to predict accurate trajectories. SemanticFormer comprises a hierarchical heterogeneous graph encoder to capture spatio-temporal and relational information across agents as well as between agents and road elements. Further, it includes a predictor to fuse different encodings and decode trajectories with probabilities. Finally, a refinement module assesses permitted meta-paths of trajectories and speed profiles to obtain final predicted trajectories. Evaluation of the nuScenes benchmark demonstrates improved performance compared to several SOTA methods. In addition, we demonstrate that our knowledge graph can be easily added to two graph-based existing SOTA methods, namely VectorNet and Laformer, replacing their original homogeneous graphs. The evaluation results suggest that by adding our knowledge graph the performance of the original methods is enhanced by 5% and 4%, respectively.

翻译：自动驾驶中的轨迹预测依赖于对驾驶场景所有相关上下文的准确表示，包括交通参与者、道路拓扑、交通标志及其相互间的语义关系。尽管该问题日益受到关注，但轨迹预测领域的大多数方法未能充分考虑所有这些因素。本文提出SemanticFormer，一种通过混合方法对语义交通场景图进行推理以预测多模态轨迹的方法。该方法利用元路径形式的高层信息（即从知识图谱中提取的允许智能体行驶的轨迹），通过基于多重注意力机制的新型处理流程来预测精确轨迹。SemanticFormer包含分层异构图编码器，用于捕捉智能体之间以及智能体与道路元素之间的时空关系和关联信息。此外，该系统还包含融合不同编码并解码带概率轨迹的预测器。最后，通过细化模块评估轨迹的允许元路径与速度剖面，获得最终预测轨迹。在nuScenes基准测试上的评估表明，其性能优于多种现有先进方法。此外，我们证明所提出的知识图谱可轻松集成至两种基于图结构的现有先进方法（VectorNet和Laformer）中，替代其原有的同质图。评估结果显示，通过引入我们的知识图谱，原始方法的性能分别提升了5%和4%。