In autonomous driving, Vehicle-Infrastructure Cooperative 3D Object Detection (VIC3D) makes use of multi-view cameras from both vehicles and traffic infrastructure, providing a global vantage point with rich semantic context of road conditions beyond a single vehicle viewpoint. Two major challenges prevail in VIC3D: 1) inherent calibration noise when fusing multi-view images, caused by time asynchrony across cameras; 2) information loss when projecting 2D features into 3D space. To address these issues, We propose a novel 3D object detection framework, Vehicles-Infrastructure Multi-view Intermediate fusion (VIMI). First, to fully exploit the holistic perspectives from both vehicles and infrastructure, we propose a Multi-scale Cross Attention (MCA) module that fuses infrastructure and vehicle features on selective multi-scales to correct the calibration noise introduced by camera asynchrony. Then, we design a Camera-aware Channel Masking (CCM) module that uses camera parameters as priors to augment the fused features. We further introduce a Feature Compression (FC) module with channel and spatial compression blocks to reduce the size of transmitted features for enhanced efficiency. Experiments show that VIMI achieves 15.61% overall AP_3D and 21.44% AP_BEV on the new VIC3D dataset, DAIR-V2X-C, significantly outperforming state-of-the-art early fusion and late fusion methods with comparable transmission cost.

翻译：在自动驾驶中，车路协同三维物体检测（VIC3D）利用车辆和交通基础设施的多视角摄像头，提供超越单一车辆视角的全局视野及丰富的道路语义上下文。VIC3D面临两大核心挑战：1）多视角图像融合时因跨摄像头时间异步引起的固有标定噪声；2）将二维特征投影至三维空间时的信息损失。为解决上述问题，我们提出一种新型三维物体检测框架——车-路多视角中间融合（VIMI）。首先，为充分挖掘车路双方的全局视角，我们设计多尺度交叉注意力（MCA）模块，在选择性多尺度上融合基础设施与车辆特征，以修正摄像头异步引入的标定噪声。其次，我们提出摄像头感知通道掩码（CCM）模块，利用摄像头参数作为先验增强融合特征。此外，引入包含通道压缩与空间压缩块的特征压缩（FC）模块，减小传输特征尺寸以提升效率。实验表明，VIMI在新型VIC3D数据集DAIR-V2X-C上实现整体AP_3D 15.61%和AP_BEV 21.44%，以可比传输代价显著超越现有最优的早期融合与晚期融合方法。