Scene reconstruction has emerged as a central challenge in computer vision, with approaches such as Neural Radiance Fields (NeRF) and Gaussian Splatting achieving remarkable progress. While Gaussian Splatting demonstrates strong performance on large-scale datasets, it often struggles to capture fine details or maintain realism in regions with sparse coverage, largely due to the inherent limitations of sparse 3D training data. In this work, we propose GauSSmart, a hybrid method that effectively bridges 2D foundational models and 3D Gaussian Splatting reconstruction. Our approach integrates established 2D computer vision techniques, including convex filtering and semantic feature supervision from foundational models such as DINO, to enhance Gaussian-based scene reconstruction. By leveraging 2D segmentation priors and high-dimensional feature embeddings, our method guides the densification and refinement of Gaussian splats, improving coverage in underrepresented areas and preserving intricate structural details. We validate our approach across three datasets, where GauSSmart consistently outperforms existing Gaussian Splatting in the majority of evaluated scenes. Our results demonstrate the significant potential of hybrid 2D-3D approaches, highlighting how the thoughtful combination of 2D foundational models with 3D reconstruction pipelines can overcome the limitations inherent in either approach alone.

翻译：场景重建已成为计算机视觉领域的核心挑战，其中神经辐射场（NeRF）与高斯溅射等方法取得了显著进展。尽管高斯溅射在大规模数据集上表现出色，但在稀疏覆盖区域往往难以捕捉精细细节或保持真实感，这主要源于稀疏3D训练数据的内在局限性。本研究提出GauSSmart——一种有效连接2D基础模型与3D高斯溅射重建的混合方法。该方法整合了成熟的2D计算机视觉技术，包括凸滤波以及来自DINO等基础模型的语义特征监督，以增强基于高斯的场景重建。通过利用2D分割先验与高维特征嵌入，我们的方法引导高斯溅射的致密化与精细化过程，在表征不足区域提升覆盖度并保留复杂结构细节。我们在三个数据集上验证了该方法，GauSSmart在多数评估场景中持续超越现有高斯溅射方法。实验结果表明了2D-3D混合方法的巨大潜力，凸显了将2D基础模型与3D重建流程进行有机结合，能够克服单一方法固有的局限性。