We present a novel method for 3D surface reconstruction from multiple images where only a part of the object of interest is captured. Our approach builds on two recent developments: surface reconstruction using neural radiance fields for the reconstruction of the visible parts of the surface, and guidance of pre-trained 2D diffusion models in the form of Score Distillation Sampling (SDS) to complete the shape in unobserved regions in a plausible manner. We introduce three components. First, we suggest employing normal maps as a pure geometric representation for SDS instead of color renderings which are entangled with the appearance information. Second, we introduce the freezing of the SDS noise during training which results in more coherent gradients and better convergence. Third, we propose Multi-View SDS as a way to condition the generation of the non-observable part of the surface without fine-tuning or making changes to the underlying 2D Stable Diffusion model. We evaluate our approach on the BlendedMVS dataset demonstrating significant qualitative and quantitative improvements over competing methods.

翻译：摘要：本文提出一种从多张仅捕获目标物体部分区域的图像进行三维表面重建的新方法。该方法基于两项近期进展：利用神经辐射场重建表面可见部分的表面重建技术，以及通过得分蒸馏采样（SDS）引导预训练二维扩散模型以合理方式补全未观测区域形状的技术。我们引入三个核心组件：首先，提出采用法线图作为SDS的纯几何表征，取代与外观信息耦合的颜色渲染图；其次，引入训练期间冻结SDS噪声的策略，该策略可生成更连贯的梯度并改善收敛性；第三，提出多视图SDS方法，无需微调或修改底层二维稳定扩散模型，即可对表面不可见部分的生成施加条件。我们在BlendedMVS数据集上评估了该方法，结果表明其相较于现有方法在定性和定量指标上均具有显著优势。