Neural 3D scene representations have shown great potential for 3D reconstruction from 2D images. However, reconstructing real-world captures of complex scenes still remains a challenge. Existing generic 3D reconstruction methods often struggle to represent fine geometric details and do not adequately model reflective surfaces of large-scale scenes. Techniques that explicitly focus on reflective surfaces can model complex and detailed reflections by exploiting better reflection parameterizations. However, we observe that these methods are often not robust in real unbounded scenarios where non-reflective as well as reflective components are present. In this work, we propose UniSDF, a general purpose 3D reconstruction method that can reconstruct large complex scenes with reflections. We investigate both view-based as well as reflection-based color prediction parameterization techniques and find that explicitly blending these representations in 3D space enables reconstruction of surfaces that are more geometrically accurate, especially for reflective surfaces. We further combine this representation with a multi-resolution grid backbone that is trained in a coarse-to-fine manner, enabling faster reconstructions than prior methods. Extensive experiments on object-level datasets DTU, Shiny Blender as well as unbounded datasets Mip-NeRF 360 and Ref-NeRF real demonstrate that our method is able to robustly reconstruct complex large-scale scenes with fine details and reflective surfaces. Please see our project page at https://fangjinhuawang.github.io/UniSDF.

翻译：神经三维场景表示在从二维图像进行三维重建方面展现出巨大潜力。然而，真实复杂场景的捕获重建仍然存在挑战。现有通用三维重建方法通常难以呈现精细几何细节，且无法充分建模大规模场景中的反射表面。专攻反射表面的技术通过利用更优的反射参数化方法可建模复杂精细的反射。但我们观察到，这些方法在同时存在非反射与反射成分的真实无界场景中往往缺乏鲁棒性。本文提出UniSDF——一种通用三维重建方法，可重建包含反射的大规模复杂场景。我们研究了基于视角与基于反射的两种颜色预测参数化技术，发现将这两种表示在三维空间中进行显式混合，能够重建出几何精度更高的表面，尤其在反射表面方面。进一步地，我们将该表示与多分辨率网格主干结合，通过由粗到细的训练方式实现比先前方法更快的重建速度。在物体级数据集DTU、Shiny Blender以及无界数据集Mip-NeRF 360与Ref-NeRF real上的大量实验表明，我们的方法能够鲁棒地重建包含精细细节与反射表面的大规模复杂场景。项目页面见https://fangjinhuawang.github.io/UniSDF。