In the field of 3D object detection for autonomous driving, LiDAR-Camera (LC) fusion is the top-performing sensor configuration. Still, LiDAR is relatively high cost, which hinders adoption of this technology for consumer automobiles. Alternatively, camera and radar are commonly deployed on vehicles already on the road today, but performance of Camera-Radar (CR) fusion falls behind LC fusion. In this work, we propose Camera-Radar Knowledge Distillation (CRKD) to bridge the performance gap between LC and CR detectors with a novel cross-modality KD framework. We use the Bird's-Eye-View (BEV) representation as the shared feature space to enable effective knowledge distillation. To accommodate the unique cross-modality KD path, we propose four distillation losses to help the student learn crucial features from the teacher model. We present extensive evaluations on the nuScenes dataset to demonstrate the effectiveness of the proposed CRKD framework. The project page for CRKD is https://song-jingyu.github.io/CRKD.

翻译：在自动驾驶三维目标检测领域，激光雷达-相机融合是性能最优的传感器配置方案。然而，激光雷达成本相对较高，阻碍了该技术在消费级汽车中的普及。相比之下，当前量产车辆普遍部署了相机和雷达，但相机-雷达融合的检测性能仍落后于激光雷达-相机融合。本文提出相机-雷达知识蒸馏框架，通过创新的跨模态知识蒸馏机制缩小两类检测器之间的性能差距。我们采用鸟瞰图表示作为共享特征空间，从而实现高效的知识蒸馏。针对跨模态知识蒸馏路径的特殊性，我们设计了四种蒸馏损失函数，帮助学生模型有效学习教师模型的关键特征。基于nuScenes数据集的广泛评估验证了所提CRKD框架的有效性。项目主页：https://song-jingyu.github.io/CRKD。