3D detection is a critical task that enables machines to identify and locate objects in three-dimensional space. It has a broad range of applications in several fields, including autonomous driving, robotics and augmented reality. Monocular 3D detection is attractive as it requires only a single camera, however, it lacks the accuracy and robustness required for real world applications. High resolution LiDAR on the other hand, can be expensive and lead to interference problems in heavy traffic given their active transmissions. We propose a balanced approach that combines the advantages of monocular and point cloud-based 3D detection. Our method requires only a small number of 3D points, that can be obtained from a low-cost, low-resolution sensor. Specifically, we use only 512 points, which is just 1% of a full LiDAR frame in the KITTI dataset. Our method reconstructs a complete 3D point cloud from this limited 3D information combined with a single image. The reconstructed 3D point cloud and corresponding image can be used by any multi-modal off-the-shelf detector for 3D object detection. By using the proposed network architecture with an off-the-shelf multi-modal 3D detector, the accuracy of 3D detection improves by 20% compared to the state-of-the-art monocular detection methods and 6% to 9% compare to the baseline multi-modal methods on KITTI and JackRabbot datasets.

翻译：三维检测是一项关键任务，使机器能够识别并定位三维空间中的物体。该技术在自动驾驶、机器人和增强现实等多个领域具有广泛应用。单目三维检测因其仅需单个摄像头而具有吸引力，但缺乏实际应用所需的准确性和鲁棒性。而高分辨率LiDAR虽能提供更高精度，但其主动发射特性可能导致成本高昂，并在交通密集场景下引发信号干扰问题。我们提出一种平衡方法，融合单目检测与基于点云的三维检测优势。该方法仅需少量三维点数据，可通过低成本、低分辨率传感器获取。具体而言，我们仅使用512个点（仅为KITTI数据集中完整LiDAR帧的1%）。通过将有限的三维信息与单幅图像相结合，我们的方法能够重构完整的3D点云。重构后的三维点云及对应图像可被任意现成的多模态检测器用于三维目标检测。采用所提出的网络架构配合现成的多模态三维检测器，在KITTI和JackRabbot数据集上，三维检测精度相较于最先进的单目检测方法提升20%，相较于基线多模态方法提升6%至9%。