Neural Radiance Fields (NeRF) are widely used for novel-view synthesis and have been adapted for 3D Object Detection (3DOD), offering a promising approach to 3DOD through view-synthesis representation. However, NeRF faces inherent limitations: (i) limited representational capacity for 3DOD due to its implicit nature, and (ii) slow rendering speeds. Recently, 3D Gaussian Splatting (3DGS) has emerged as an explicit 3D representation that addresses these limitations. Inspired by these advantages, this paper introduces 3DGS into 3DOD for the first time, identifying two main challenges: (i) Ambiguous spatial distribution of Gaussian blobs: 3DGS primarily relies on 2D pixel-level supervision, resulting in unclear 3D spatial distribution of Gaussian blobs and poor differentiation between objects and background, which hinders 3DOD; (ii) Excessive background blobs: 2D images often include numerous background pixels, leading to densely reconstructed 3DGS with many noisy Gaussian blobs representing the background, negatively affecting detection. To tackle the challenge (i), we leverage the fact that 3DGS reconstruction is derived from 2D images, and propose an elegant and efficient solution by incorporating 2D Boundary Guidance to significantly enhance the spatial distribution of Gaussian blobs, resulting in clearer differentiation between objects and their background. To address the challenge (ii), we propose a Box-Focused Sampling strategy using 2D boxes to generate object probability distribution in 3D spaces, allowing effective probabilistic sampling in 3D to retain more object blobs and reduce noisy background blobs. Benefiting from our designs, our 3DGS-DET significantly outperforms the SOTA NeRF-based method, NeRF-Det, achieving improvements of +6.6 on mAP@0.25 and +8.1 on mAP@0.5 for the ScanNet dataset, and impressive +31.5 on mAP@0.25 for the ARKITScenes dataset.

翻译：神经辐射场（NeRF）被广泛用于新视角合成，并已适配于三维目标检测（3DOD），为通过视角合成表征实现3DOD提供了有前景的途径。然而，NeRF存在固有局限：（i）因其隐式表征特性，对3DOD的表征能力有限；（ii）渲染速度缓慢。近期，3D高斯溅射（3DGS）作为一种显式三维表征方法出现，解决了这些局限。受这些优势启发，本文首次将3DGS引入3DOD领域，并识别出两大挑战：（i）高斯斑点的空间分布模糊性：3DGS主要依赖二维像素级监督，导致高斯斑点的三维空间分布不明确，目标与背景区分度差，从而阻碍3DOD；（ii）过量背景斑点：二维图像常包含大量背景像素，导致重建的3DGS密集且存在大量表征背景的噪声高斯斑点，对检测产生负面影响。为应对挑战（i），我们利用3DGS重建源于二维图像的特性，提出一种优雅高效的解决方案——引入二维边界引导，显著增强高斯斑点的空间分布，从而实现目标与背景的更清晰区分。为解决挑战（ii），我们提出一种聚焦边界框采样策略，利用二维边界框在三维空间中生成目标概率分布，实现在三维空间中进行有效的概率采样，以保留更多目标斑点并减少噪声背景斑点。得益于我们的设计，所提出的3DGS-DET显著优于基于NeRF的SOTA方法NeRF-Det：在ScanNet数据集上mAP@0.25提升+6.6，mAP@0.5提升+8.1；在ARKITScenes数据集上mAP@0.25实现显著提升+31.5。