With the rapid development of Pattern Recognition and Computer Vision technologies, tasks like object detection or semantic segmentation have achieved even better accuracy than human beings. Based on these solid foundations, autonomous driving is becoming an important research direction, aiming to revolute the future of transportation and mobility. Sensors are critical to autonomous driving's security and feasibility to perceive the surrounding environment. Multi-Sensor fusion has become a current research hot spot because of its potential for multidimensional perception and integration ability. In this paper, we propose a novel feature-level multi-sensor fusion technology for end-to-end autonomous driving navigation with imitation learning. Our paper mainly focuses on fusion technologies for Lidar and RGB information. We also provide a brand-new penalty-based imitation learning method to reinforce the model's compliance with traffic rules and unify the objective of imitation learning and the metric of autonomous driving.

翻译：随着模式识别与计算机视觉技术的快速发展，物体检测、语义分割等任务已取得超越人类精度的优异表现。基于这些坚实基础，自动驾驶正成为旨在革新未来交通出行方式的重要研究方向。传感器对自动驾驶的安全性与可行性至关重要，用以感知周围环境。多传感器融合凭借其多维感知与集成能力的潜力，已成为当前研究热点。本文提出一种新颖的特征级多传感器融合技术，用于基于模仿学习的端到端自动驾驶导航。研究重点聚焦于激光雷达与RGB信息融合技术。同时，我们创新性地提出基于惩罚的模仿学习方法，以增强模型对交通规则的遵从性，统一模仿学习的目标与自动驾驶的评估标准。