Gaussian Splatting (GS)-based methods rely on sufficient training view coverage and perform synthesis on interpolated views. In this work, we tackle the more challenging and underexplored Extrapolated View Synthesis (EVS) task. Here we enable GS-based models trained with limited view coverage to generalize well to extrapolated views. To achieve our goal, we propose a view augmentation framework to guide training through a coarse-to-fine process. At the coarse stage, we reduce rendering artifacts due to insufficient view coverage by introducing a regularization strategy at both appearance and geometry levels. At the fine stage, we generate reliable view priors to provide further training guidance. To this end, we incorporate an occlusion awareness into the view prior generation process, and refine the view priors with the aid of coarse stage output. We call our framework Enhanced View Prior Guidance for Splatting (EVPGS). To comprehensively evaluate EVPGS on the EVS task, we collect a real-world dataset called Merchandise3D dedicated to the EVS scenario. Experiments on three datasets including both real and synthetic demonstrate EVPGS achieves state-of-the-art performance, while improving synthesis quality at extrapolated views for GS-based methods both qualitatively and quantitatively. We will make our code, dataset, and models public.
翻译:基于高斯溅射(GS)的方法依赖于充分的训练视角覆盖,并在插值视角上进行合成。在本研究中,我们致力于解决更具挑战性且尚未充分探索的外推视图合成(EVS)任务。在此,我们使基于GS的模型在有限视角覆盖下训练后,能够良好地泛化至外推视角。为实现这一目标,我们提出了一个视角增强框架,通过从粗到精的过程指导训练。在粗粒度阶段,我们通过在表观和几何层面引入正则化策略,减少因视角覆盖不足导致的渲染伪影。在精粒度阶段,我们生成可靠的视角先验以提供进一步的训练指导。为此,我们在视角先验生成过程中融入了遮挡感知,并借助粗粒度阶段的输出对视角先验进行细化。我们将该框架称为增强视角先验引导溅射(EVPGS)。为全面评估EVPGS在EVS任务上的性能,我们收集了一个专用于EVS场景的真实世界数据集Merchandise3D。在包含真实和合成数据在内的三个数据集上的实验表明,EVPGS实现了最先进的性能,同时在定性和定量上均提升了基于GS的方法在外推视角处的合成质量。我们将公开代码、数据集和模型。