Reconstructing and rendering 3D objects from highly sparse views is of critical importance for promoting applications of 3D vision techniques and improving user experience. However, images from sparse views only contain very limited 3D information, leading to two significant challenges: 1) Difficulty in building multi-view consistency as images for matching are too few; 2) Partially omitted or highly compressed object information as view coverage is insufficient. To tackle these challenges, we propose GaussianObject, a framework to represent and render the 3D object with Gaussian splatting, that achieves high rendering quality with only 4 input images. We first introduce techniques of visual hull and floater elimination which explicitly inject structure priors into the initial optimization process for helping build multi-view consistency, yielding a coarse 3D Gaussian representation. Then we construct a Gaussian repair model based on diffusion models to supplement the omitted object information, where Gaussians are further refined. We design a self-generating strategy to obtain image pairs for training the repair model. Our GaussianObject is evaluated on several challenging datasets, including MipNeRF360, OmniObject3D, and OpenIllumination, achieving strong reconstruction results from only 4 views and significantly outperforming previous state-of-the-art methods.

翻译：从高度稀疏视角重建并渲染三维物体，对于推动三维视觉技术应用和提升用户体验具有关键意义。然而，稀疏视角图像仅包含极为有限的三维信息，导致两大挑战：1）因匹配图像过少而难以构建多视角一致性；2）因视角覆盖不足导致物体信息部分缺失或高度压缩。为应对这些挑战，我们提出GaussianObject框架——基于高斯点绘（Gaussian Splatting）进行三维物体表示与渲染，仅需4张输入图像即可实现高质量渲染。我们首先引入视觉外壳与浮动点消除技术，通过向初始优化过程显式注入结构先验以构建多视角一致性，获得粗糙的三维高斯表示。随后基于扩散模型构建高斯修复模型，对缺失物体信息进行补充，并进一步优化高斯体。我们设计自生成策略获取用于训练修复模型的图像对。在MipNeRF360、OmniObject3D和OpenIllumination等多个挑战性数据集上的评估表明，本方法仅凭4个视角即能实现强重建效果，显著超越现有最优方法。