More research attention has recently been given to end-to-end autonomous driving technologies where the entire driving pipeline is replaced with a single neural network because of its simpler structure and faster inference time. Despite this appealing approach largely reducing the components in driving pipeline, its simplicity also leads to interpretability problems and safety issues arXiv:2003.06404. The trained policy is not always compliant with the traffic rules and it is also hard to discover the reason for the misbehavior because of the lack of intermediate outputs. Meanwhile, Sensors are also critical to autonomous driving's security and feasibility to perceive the surrounding environment under complex driving scenarios. In this paper, we proposed P-CSG, a novel penalty-based imitation learning approach with cross semantics generation sensor fusion technologies to increase the overall performance of End-to-End Autonomous Driving. We conducted an assessment of our model's performance using the Town 05 Long benchmark, achieving an impressive driving score improvement of over 15%. Furthermore, we conducted robustness evaluations against adversarial attacks like FGSM and Dot attacks, revealing a substantial increase in robustness compared to baseline models.More detailed information, such as code-based resources, ablation studies and videos can be found at https://hk-zh.github.io/p-csg-plus.

翻译：近年来，端到端自动驾驶技术因其结构简单、推理速度快而受到更多研究关注，该技术将整个驾驶流程替换为单个神经网络。尽管这一方法大幅减少了驾驶流水线中的组件，但其简洁性也带来了可解释性问题和安全隐患（arXiv:2003.06404）。训练得到的策略并非始终符合交通规则，且由于缺乏中间输出，难以发现违规行为的原因。同时，传感器对于自动驾驶在复杂驾驶场景中感知周围环境的安全性和可行性也至关重要。本文提出了一种名为P-CSG的新型基于惩罚的模仿学习方法，结合跨语义生成传感器融合技术，以提升端到端自动驾驶的整体性能。我们使用Town 05 Long基准评估模型性能，取得了超过15%的显著驾驶分数提升。此外，针对FGSM和点攻击等对抗性攻击的鲁棒性评估表明，与基线模型相比，鲁棒性显著增强。更多详细信息，如基于代码的资源、消融研究和视频，请访问https://hk-zh.github.io/p-csg-plus。