In recent times, there has been a growing focus on end-to-end autonomous driving technologies. This technology involves the replacement of the entire driving pipeline with a single neural network, which has a simpler structure and faster inference time. However, while this approach reduces the number of components in the driving pipeline, it also presents challenges related to interpretability and safety. For instance, the trained policy may not always comply with traffic rules, and it is difficult to determine the reason for such misbehavior due to the lack of intermediate outputs. Additionally, the successful implementation of autonomous driving technology heavily depends on the reliable and expedient processing of sensory data to accurately perceive the surrounding environment. In this paper, we provide penalty-based imitation learning approach combined with cross semantics generation sensor fusion technologies (P-CSG) to efficiently integrate multiple modalities of information and enable the autonomous agent to effectively adhere to traffic regulations. Our model undergoes evaluation within the Town 05 Long benchmark, where we observe a remarkable increase in the driving score by more than 12% when compared to the state-of-the-art (SOTA) model, InterFuser. Notably, our model achieves this performance enhancement while achieving a 7-fold increase in inference speed and reducing the model size by approximately 30%. For more detailed information, including code-based resources, they can be found at https://hk-zh.github.io/p-csg/

翻译：近年来，端到端自主驾驶技术日益受到关注。该技术通过单一神经网络替代整个驾驶管线，具有结构更简单、推理速度更快的优势。然而，这种简化方法在减少驾驶管线组件数量的同时，也带来了可解释性与安全性方面的挑战。例如，训练得到的策略可能无法始终遵守交通规则，且由于缺乏中间输出结果，难以判断违规行为的具体原因。此外，自主驾驶技术的成功实施高度依赖于对传感器数据的可靠、高效处理，以实现对周围环境的精确感知。本文提出一种结合跨语义生成传感器融合技术的基于惩罚的模仿学习方法（P-CSG），旨在高效整合多模态信息，使自主智能体有效遵守交通规则。在Town 05 Long基准测试中，我们的模型相较于当前最优模型InterFuser，驾驶得分提升超过12%。值得注意的是，在实现该性能提升的同时，模型推理速度提高7倍，模型体积减小约30%。更多详细信息（包括代码资源）可访问https://hk-zh.github.io/p-csg/获取。