Generalized feed-forward Gaussian models have achieved significant progress in sparse-view 3D reconstruction by leveraging prior knowledge from large multi-view datasets. However, these models often struggle to represent high-frequency details due to the limited number of Gaussians. While the densification strategy used in per-scene 3D Gaussian splatting (3D-GS) optimization can be adapted to the feed-forward models, it may not be ideally suited for generalized scenarios. In this paper, we propose Generative Densification, an efficient and generalizable method to densify Gaussians generated by feed-forward models. Unlike the 3D-GS densification strategy, which iteratively splits and clones raw Gaussian parameters, our method up-samples feature representations from the feed-forward models and generates their corresponding fine Gaussians in a single forward pass, leveraging the embedded prior knowledge for enhanced generalization. Experimental results on both object-level and scene-level reconstruction tasks demonstrate that our method outperforms state-of-the-art approaches with comparable or smaller model sizes, achieving notable improvements in representing fine details.
翻译:基于前馈的高斯模型通过利用大型多视角数据集的先验知识,在稀疏视角三维重建领域取得了显著进展。然而,由于高斯元素数量有限,这些模型往往难以表征高频细节。虽然逐场景三维高斯溅射优化中使用的致密化策略可以适配于前馈模型,但其可能并不完全适用于泛化场景。本文提出生成式致密化方法,这是一种高效且可泛化的技术,用于对前馈模型生成的高斯元素进行致密化。与三维高斯溅射通过迭代分裂和克隆原始高斯参数的致密化策略不同,我们的方法通过单次前向传播,对前馈模型的特征表示进行上采样,并生成对应的精细高斯元素,同时利用嵌入的先验知识以增强泛化能力。在物体级和场景级重建任务上的实验结果表明,本方法在模型参数量相当或更小的条件下,超越了现有最优方法,在精细细节表征方面取得了显著提升。