In this paper, we propose a novel method for 3D scene and object reconstruction from sparse multi-view images. Different from previous methods that leverage extra information such as depth or generalizable features across scenes, our approach leverages the scene properties embedded in the multi-view inputs to create precise pseudo-labels for optimization without any prior training. Specifically, we introduce a geometry-guided approach that improves surface reconstruction accuracy from sparse views by leveraging spherical harmonics to predict the novel radiance while holistically considering all color observations for a point in the scene. Also, our pipeline exploits proxy geometry and correctly handles the occlusion in generating the pseudo-labels of radiance, which previous image-warping methods fail to avoid. Our method, dubbed Ray Augmentation (RayAug), achieves superior results on DTU and Blender datasets without requiring prior training, demonstrating its effectiveness in addressing the problem of sparse view reconstruction. Our pipeline is flexible and can be integrated into other implicit neural reconstruction methods for sparse views.

翻译：本文提出一种新颖的三维场景与物体重建方法，仅需稀疏多视角图像即可实现重建。与依赖深度信息或跨场景通用特征等额外先验信息的现有方法不同，本方法利用多视角输入中蕴含的场景属性，在无需任何预训练的情况下生成精确伪标签进行优化。具体而言，我们提出几何引导策略：通过球谐函数预测新视角辐射场，并全局考虑场景中某点所有颜色观测值，从而提升稀疏视角下的表面重建精度。同时，我们的流程利用代理几何正确解决遮挡问题，生成辐射伪标签——这一难题是现有图像变形方法无法避免的。本方法命名为射线增强（RayAug），在DTU和Blender数据集上无需预训练即取得优越性能，充分验证其对稀疏视角重建问题的有效性。该流程具有高度灵活性，可集成至其他隐式神经重建方法中处理稀疏视角场景。