LiDAR-based vision systems are integral for 3D object detection, which is crucial for autonomous navigation. However, they suffer from performance degradation in adverse weather conditions due to the quality deterioration of LiDAR point clouds. Fusing LiDAR with the weather-robust 4D radar sensor is expected to solve this problem. However, the fusion of LiDAR and 4D radar is challenging because they differ significantly in terms of data quality and the degree of degradation in adverse weather. To address these issues, we introduce L4DR, a weather-robust 3D object detection method that effectively achieves LiDAR and 4D Radar fusion. Our L4DR includes Multi-Modal Encoding (MME) and Foreground-Aware Denoising (FAD) technique to reconcile sensor gaps, which is the first exploration of the complementarity of early fusion between LiDAR and 4D radar. Additionally, we design an Inter-Modal and Intra-Modal ({IM}2 ) parallel feature extraction backbone coupled with a Multi-Scale Gated Fusion (MSGF) module to counteract the varying degrees of sensor degradation under adverse weather conditions. Experimental evaluation on a VoD dataset with simulated fog proves that L4DR is more adaptable to changing weather conditions. It delivers a significant performance increase under different fog levels, improving the 3D mAP by up to 20.0% over the traditional LiDAR-only approach. Moreover, the results on the K-Radar dataset validate the consistent performance improvement of L4DR in real-world adverse weather conditions.

翻译：基于激光雷达的视觉系统是三维目标检测的关键组成部分，对自主导航至关重要。然而，在恶劣天气条件下，由于激光雷达点云质量下降，其性能会显著退化。将激光雷达与具有天气鲁棒性的4D雷达传感器融合有望解决这一问题。然而，激光雷达与4D雷达的融合面临挑战，因为二者在数据质量及恶劣天气下的退化程度上存在显著差异。为解决这些问题，我们提出L4DR——一种天气鲁棒性三维目标检测方法，有效实现了激光雷达与4D雷达的融合。L4DR包含多模态编码（MME）与前景感知去噪（FAD）技术，以弥合传感器差异，这是首次探索激光雷达与4D雷达早期融合互补性的研究。此外，我们设计了一种结合跨模态与模态内（{IM}2）并行特征提取主干网络的多尺度门控融合（MSGF）模块，以应对恶劣天气下不同程度的传感器性能退化。在带有模拟雾的VoD数据集上的实验评估证明，L4DR对变化天气条件具有更强的适应能力。在不同雾浓度下均实现了显著的性能提升，相比传统纯激光雷达方法，三维平均精度（mAP）最高提升20.0%。此外，在K-Radar数据集上的结果验证了L4DR在真实恶劣天气条件下持续的性能改进。