Neural radiance fields (NeRFs) have recently emerged as a promising approach for 3D reconstruction and novel view synthesis. However, NeRF-based methods encode shape, reflectance, and illumination implicitly and this makes it challenging for users to manipulate these properties in the rendered images explicitly. Existing approaches only enable limited editing of the scene and deformation of the geometry. Furthermore, no existing work enables accurate scene illumination after object deformation. In this work, we introduce SPIDR, a new hybrid neural SDF representation. SPIDR combines point cloud and neural implicit representations to enable the reconstruction of higher quality object surfaces for geometry deformation and lighting estimation. meshes and surfaces for object deformation and lighting estimation. To more accurately capture environment illumination for scene relighting, we propose a novel neural implicit model to learn environment light. To enable more accurate illumination updates after deformation, we use the shadow mapping technique to approximate the light visibility updates caused by geometry editing. We demonstrate the effectiveness of SPIDR in enabling high quality geometry editing with more accurate updates to the illumination of the scene.

翻译：神经辐射场（NeRF）近期作为三维重建和新视角合成的有效方法而兴起。然而，基于NeRF的方法隐式编码了形状、反射率和光照信息，这使得用户难以在渲染图像中显式操控这些属性。现有方法仅能对场景进行有限编辑，并对几何结构实现受限的形变处理。此外，尚无现有工作能在物体形变后实现精确的场景光照。本文提出SPIDR——一种新型混合神经有符号距离函数（SDF）表示方法。SPIDR融合点云与神经隐式表示，可重建更高质量物体表面以支持几何形变与光照估计。为更精确地捕捉环境光照以实现场景重打光，我们提出一种新型神经隐式模型来学习环境光。为实现形变后更精确的光照更新，我们采用阴影映射技术近似几何编辑引起的光可见性变化。实验证明，SPIDR能实现高质量几何编辑，并对场景光照进行更精确的更新。