Reconstructing the shape and spatially varying surface appearances of a physical-world object as well as its surrounding illumination based on 2D images (e.g., photographs) of the object has been a long-standing problem in computer vision and graphics. In this paper, we introduce an accurate and highly efficient object reconstruction pipeline combining neural based object reconstruction and physics-based inverse rendering (PBIR). Our pipeline firstly leverages a neural SDF based shape reconstruction to produce high-quality but potentially imperfect object shape. Then, we introduce a neural material and lighting distillation stage to achieve high-quality predictions for material and illumination. In the last stage, initialized by the neural predictions, we perform PBIR to refine the initial results and obtain the final high-quality reconstruction of object shape, material, and illumination. Experimental results demonstrate our pipeline significantly outperforms existing methods quality-wise and performance-wise.

翻译：从物理世界物体的二维图像（例如照片）重建其形状、空间变化表面外观以及周围光照，一直是计算机视觉与图形学领域的长期难题。本文提出了一种融合基于神经的物体重建与基于物理的逆渲染（PBIR）的精确高效物体重建流水线。该流水线首先利用基于神经符号距离函数（SDF）的形状重建，生成高质量但可能不完美的物体形状；随后引入神经材质与光照蒸馏阶段，实现材质与光照的高质量预测；最后阶段通过神经预测初始化，执行PBIR以优化初始结果，获得物体形状、材质与光照的最终高质量重建。实验结果表明，本流水线在质量与性能上显著优于现有方法。