As one of the automotive sensors that have emerged in recent years, 4D millimeter-wave radar has a higher resolution than conventional 3D radar and provides precise elevation measurements. But its point clouds are still sparse and noisy, making it challenging to meet the requirements of autonomous driving. Camera, as another commonly used sensor, can capture rich semantic information. As a result, the fusion of 4D radar and camera can provide an affordable and robust perception solution for autonomous driving systems. However, previous radar-camera fusion methods have not yet been thoroughly investigated, resulting in a large performance gap compared to LiDAR-based methods. Specifically, they ignore the feature-blurring problem and do not deeply interact with image semantic information. To this end, we present a simple but effective multi-stage sampling fusion (MSSF) network based on 4D radar and camera. On the one hand, we design a fusion block that can deeply interact point cloud features with image features, and can be applied to commonly used single-modal backbones in a plug-and-play manner. The fusion block encompasses two types, namely, simple feature fusion (SFF) and multiscale deformable feature fusion (MSDFF). The SFF is easy to implement, while the MSDFF has stronger fusion abilities. On the other hand, we propose a semantic-guided head to perform foreground-background segmentation on voxels with voxel feature re-weighting, further alleviating the problem of feature blurring. Extensive experiments on the View-of-Delft (VoD) and TJ4DRadset datasets demonstrate the effectiveness of our MSSF. Notably, compared to state-of-the-art methods, MSSF achieves a 7.0% and 4.0% improvement in 3D mean average precision on the VoD and TJ4DRadSet datasets, respectively. It even surpasses classical LiDAR-based methods on the VoD dataset.

翻译：作为近年来出现的车载传感器之一，4D毫米波雷达比传统的3D雷达具有更高的分辨率，并能提供精确的高度测量。但其点云仍然稀疏且含有噪声，难以满足自动驾驶的要求。相机作为另一种常用传感器，能够捕获丰富的语义信息。因此，4D雷达与相机的融合可以为自动驾驶系统提供一种经济且鲁棒的感知解决方案。然而，以往的雷达-相机融合方法尚未得到深入研究，导致其性能与基于激光雷达的方法存在较大差距。具体而言，它们忽视了特征模糊问题，且未与图像语义信息进行深度交互。为此，我们提出了一种基于4D雷达与相机的、简单而有效的多阶段采样融合（MSSF）网络。一方面，我们设计了一个融合模块，能够深度交互点云特征与图像特征，并且可以以即插即用的方式应用于常用的单模态骨干网络。该融合模块包含两种类型，即简单特征融合（SFF）和多尺度可变形特征融合（MSDFF）。SFF易于实现，而MSDFF则具有更强的融合能力。另一方面，我们提出了一个语义引导头，用于对体素进行前景-背景分割并重新加权体素特征，从而进一步缓解特征模糊问题。在View-of-Delft（VoD）和TJ4DRadset数据集上进行的大量实验证明了我们MSSF的有效性。值得注意的是，与最先进的方法相比，MSSF在VoD和TJ4DRadSet数据集上的3D平均精度均值分别提高了7.0%和4.0%。在VoD数据集上，它甚至超越了经典的基于激光雷达的方法。