Learning contextual and spatial environmental representations enhances autonomous vehicle's hazard anticipation and decision-making in complex scenarios. Recent perception systems enhance spatial understanding with sensor fusion but often lack full environmental context. Humans, when driving, naturally employ neural maps that integrate various factors such as historical data, situational subtleties, and behavioral predictions of other road users to form a rich contextual understanding of their surroundings. This neural map-based comprehension is integral to making informed decisions on the road. In contrast, even with their significant advancements, autonomous systems have yet to fully harness this depth of human-like contextual understanding. Motivated by this, our work draws inspiration from human driving patterns and seeks to formalize the sensor fusion approach within an end-to-end autonomous driving framework. We introduce a framework that integrates three cameras (left, right, and center) to emulate the human field of view, coupled with top-down bird-eye-view semantic data to enhance contextual representation. The sensor data is fused and encoded using a self-attention mechanism, leading to an auto-regressive waypoint prediction module. We treat feature representation as a sequential problem, employing a vision transformer to distill the contextual interplay between sensor modalities. The efficacy of the proposed method is experimentally evaluated in both open and closed-loop settings. Our method achieves displacement error by 0.67m in open-loop settings, surpassing current methods by 6.9% on the nuScenes dataset. In closed-loop evaluations on CARLA's Town05 Long and Longest6 benchmarks, the proposed method enhances driving performance, route completion, and reduces infractions.

翻译：学习上下文与空间环境表征能增强自动驾驶车辆在复杂场景中的危险预判与决策能力。现有感知系统通过传感器融合提升空间理解，但往往缺乏完整的环境上下文信息。人类驾驶员在驾驶过程中会自然运用神经图，整合历史数据、情境细节及其他道路使用者行为预测等多种因素，形成对周边环境的丰富上下文理解。这种基于神经图的认知对于在道路上做出明智决策至关重要。相比之下，尽管自动驾驶系统取得重大进展，但仍未能完全实现类似人类的深度上下文理解能力。受此启发，本研究借鉴人类驾驶模式，致力于在端到端自动驾驶框架内形式化传感器融合方法。我们提出一种集成三个摄像头（左、右、中）以模拟人类视野的框架，并结合自上而下的鸟瞰语义数据增强上下文表征。传感器数据通过自注意力机制进行融合与编码，进而形成自回归式路径点预测模块。我们将特征表征视为序列问题，采用视觉变换器（Vision Transformer）提炼传感器模态间的上下文交互关系。所提方法的有效性在开环与闭环两种场景下均经过实验验证。在开环场景下，本方法在nuScenes数据集上实现了0.67米的位移误差，较现有方法提升6.9%。在CARLA平台Town05 Long与Longest6基准测试的闭环评估中，所提方法提升了驾驶性能、路径完成度，并减少了违章行为。