Despite the substantial progress of novel view synthesis, existing methods, either based on the Neural Radiance Fields (NeRF) or more recently 3D Gaussian Splatting (3DGS), suffer significant degradation when the input becomes sparse. Numerous efforts have been introduced to alleviate this problem, but they still struggle to synthesize satisfactory results efficiently, especially in the large scene. In this paper, we propose SCGaussian, a Structure Consistent Gaussian Splatting method using matching priors to learn 3D consistent scene structure. Considering the high interdependence of Gaussian attributes, we optimize the scene structure in two folds: rendering geometry and, more importantly, the position of Gaussian primitives, which is hard to be directly constrained in the vanilla 3DGS due to the non-structure property. To achieve this, we present a hybrid Gaussian representation. Besides the ordinary non-structure Gaussian primitives, our model also consists of ray-based Gaussian primitives that are bound to matching rays and whose optimization of their positions is restricted along the ray. Thus, we can utilize the matching correspondence to directly enforce the position of these Gaussian primitives to converge to the surface points where rays intersect. Extensive experiments on forward-facing, surrounding, and complex large scenes show the effectiveness of our approach with state-of-the-art performance and high efficiency. Code is available at https://github.com/prstrive/SCGaussian.

翻译：尽管新视角合成领域已取得显著进展，但现有方法——无论是基于神经辐射场（NeRF）还是近期兴起的3D高斯溅射（3DGS）——在输入图像稀疏时性能均会显著下降。已有大量研究致力于缓解此问题，但它们仍难以高效合成令人满意的结果，尤其是在大规模场景中。本文提出SCGaussian，一种基于匹配先验的结构一致高斯溅射方法，通过学习三维一致的场景结构来解决该问题。考虑到高斯属性间的高度相互依赖性，我们从两个层面优化场景结构：渲染几何体，以及更关键的高斯基元位置优化——后者在原始3DGS中因其非结构性而难以直接约束。为此，我们提出一种混合高斯表示方法。除常规的非结构高斯基元外，我们的模型还包含与匹配光线绑定的射线型高斯基元，其位置优化被限制沿射线方向进行。因此，我们可以利用匹配对应关系直接约束这些高斯基元的位置，使其收敛至光线相交的表面点。在前向场景、环绕场景及复杂大规模场景上的大量实验表明，我们的方法在实现最先进性能和高效率的同时具有显著有效性。代码发布于https://github.com/prstrive/SCGaussian。