We present a neural rendering-based method called NeRO for reconstructing the geometry and the BRDF of reflective objects from multiview images captured in an unknown environment. Multiview reconstruction of reflective objects is extremely challenging because specular reflections are view-dependent and thus violate the multiview consistency, which is the cornerstone for most multiview reconstruction methods. Recent neural rendering techniques can model the interaction between environment lights and the object surfaces to fit the view-dependent reflections, thus making it possible to reconstruct reflective objects from multiview images. However, accurately modeling environment lights in the neural rendering is intractable, especially when the geometry is unknown. Most existing neural rendering methods, which can model environment lights, only consider direct lights and rely on object masks to reconstruct objects with weak specular reflections. Therefore, these methods fail to reconstruct reflective objects, especially when the object mask is not available and the object is illuminated by indirect lights. We propose a two-step approach to tackle this problem. First, by applying the split-sum approximation and the integrated directional encoding to approximate the shading effects of both direct and indirect lights, we are able to accurately reconstruct the geometry of reflective objects without any object masks. Then, with the object geometry fixed, we use more accurate sampling to recover the environment lights and the BRDF of the object. Extensive experiments demonstrate that our method is capable of accurately reconstructing the geometry and the BRDF of reflective objects from only posed RGB images without knowing the environment lights and the object masks. Codes and datasets are available at https://github.com/liuyuan-pal/NeRO.

翻译：摘要：我们提出了一种名为NeRO的神经渲染方法，用于在未知环境下从多视图图像中重建反射物体的几何形状与双向反射分布函数。反射物体的多视图重建极具挑战性，因为镜面反射具有视角依赖性，这违背了大多数多视图重建方法的核心基础——多视图一致性。近年来，神经渲染技术能够模拟环境光与物体表面的相互作用以拟合视角依赖的反射，从而使得从多视图图像重建反射物体成为可能。然而，在神经渲染中精确建模环境光极其困难，尤其当几何形状未知时。现有能建模环境光的神经渲染方法大多仅考虑直接光照，并依赖物体掩码来重建具有弱镜面反射的物体。因此，这些方法无法重建反射物体，尤其在无物体掩码且物体受间接光照影响的情况下。我们提出了一种两步法来解决该问题。首先，通过应用分裂和近似与集成方向编码来近似直接光照与间接光照的着色效果，我们能够在无需任何物体掩码的情况下精确重建反射物体的几何结构。随后，在固定物体几何的基础上，我们利用更精确的采样来恢复环境光与物体的BRDF。大量实验证明，我们的方法能够仅基于已知姿态的RGB图像（无需环境光与物体掩码信息）精确重建反射物体的几何形状与BRDF。代码与数据集已开源在https://github.com/liuyuan-pal/NeRO。