With the rapid development of Pattern Recognition and Computer Vision technologies, tasks like object detection or semantic segmentation have achieved even better accuracy than human beings. Based on these solid foundations, autonomous driving is becoming an important research direction, aiming to revolute the future of transportation and mobility. Sensors are critical to autonomous driving's security and feasibility to perceive the surrounding environment. Multi-Sensor fusion has become a current research hot spot because of its potential for multidimensional perception and integration ability. In this paper, we propose a novel feature-level multi-sensor fusion technology for end-to-end autonomous driving navigation with imitation learning. Our paper mainly focuses on fusion technologies for Lidar and RGB information. We also provide a brand-new penalty-based imitation learning method to reinforce the model's compliance with traffic rules and unify the objective of imitation learning and the metric of autonomous driving.

翻译：随着模式识别与计算机视觉技术的快速发展，目标检测和语义分割等任务的精度已超越人类水平。在这些坚实基础上，自动驾驶正成为致力于革新未来交通与出行方式的重要研究方向。传感器作为自动驾驶安全性与可行性的关键，负责感知周围环境。多传感器融合凭借其多维感知与信息整合能力，已成为当前研究热点。本文提出一种新颖的特征级多传感器融合技术，用于基于模仿学习的端到端自动驾驶导航，重点研究激光雷达与RGB信息融合技术。同时，我们创新性地引入基于惩罚的模仿学习方法，以增强模型对交通规则的遵守能力，并统一模仿学习目标与自动驾驶评估指标。