Denoising diffusion models show remarkable performances in generative tasks, and their potential applications in perception tasks are gaining interest. In this paper, we introduce a novel framework named DiffRef3D which adopts the diffusion process on 3D object detection with point clouds for the first time. Specifically, we formulate the proposal refinement stage of two-stage 3D object detectors as a conditional diffusion process. During training, DiffRef3D gradually adds noise to the residuals between proposals and target objects, then applies the noisy residuals to proposals to generate hypotheses. The refinement module utilizes these hypotheses to denoise the noisy residuals and generate accurate box predictions. In the inference phase, DiffRef3D generates initial hypotheses by sampling noise from a Gaussian distribution as residuals and refines the hypotheses through iterative steps. DiffRef3D is a versatile proposal refinement framework that consistently improves the performance of existing 3D object detection models. We demonstrate the significance of DiffRef3D through extensive experiments on the KITTI benchmark. Code will be available.

翻译：摘要：去噪扩散模型在生成任务中展现出卓越性能，其在感知任务中的潜在应用正引发广泛关注。本文首次提出名为DiffRef3D的创新框架，将扩散过程应用于基于点云的三维目标检测。具体而言，我们将两阶段三维目标检测器的提案精炼阶段建模为条件扩散过程。训练过程中，DiffRef3D逐步向提案与目标物体之间的残差添加噪声，随后将含噪残差应用于提案以生成假设。精炼模块利用这些假设对含噪残差进行去噪，生成精确的边界框预测。推理阶段，DiffRef3D通过从高斯分布采样噪声作为残差生成初始假设，并迭代优化假设。DiffRef3D是一个通用型提案精炼框架，可显著提升现有三维目标检测模型的性能。通过在KITTI基准上的大量实验，我们验证了DiffRef3D的重要性。代码将开源。