3D perception based on the representations learned from multi-camera bird's-eye-view (BEV) is trending as cameras are cost-effective for mass production in autonomous driving industry. However, there exists a distinct performance gap between multi-camera BEV and LiDAR based 3D object detection. One key reason is that LiDAR captures accurate depth and other geometry measurements, while it is notoriously challenging to infer such 3D information from merely image input. In this work, we propose to boost the representation learning of a multi-camera BEV based student detector by training it to imitate the features of a well-trained LiDAR based teacher detector. We propose effective balancing strategy to enforce the student to focus on learning the crucial features from the teacher, and generalize knowledge transfer to multi-scale layers with temporal fusion. We conduct extensive evaluations on multiple representative models of multi-camera BEV. Experiments reveal that our approach renders significant improvement over the student models, leading to the state-of-the-art performance on the popular benchmark nuScenes.

翻译：基于多相机鸟瞰图（BEV）表示学习的3D感知正成为趋势，因为摄像头在自动驾驶工业的大规模生产中具有成本效益。然而，多相机BEV与基于LiDAR的3D目标检测之间存在显著的性能差距。关键原因在于LiDAR能够捕获准确的深度及其他几何测量信息，而仅从图像输入推断此类3D信息极具挑战性。本文提出通过训练基于多相机BEV的学生检测器模仿训练成熟的LiDAR教师检测器的特征，以提升其表示学习能力。我们提出有效的平衡策略，强制学生模型专注于从教师模型学习关键特征，并通过时序融合将知识迁移推广至多尺度层。我们对多种代表性多相机BEV模型进行广泛评估。实验表明，我们的方法显著提升了学生模型的性能，在主流基准数据集nuScenes上达到了最先进水平。