Reconstructing the shape and spatially varying surface appearances of a physical-world object as well as its surrounding illumination based on 2D images (e.g., photographs) of the object has been a long-standing problem in computer vision and graphics. In this paper, we introduce a robust object reconstruction pipeline combining neural based object reconstruction and physics-based inverse rendering (PBIR). Specifically, our pipeline firstly leverages a neural stage to produce high-quality but potentially imperfect predictions of object shape, reflectance, and illumination. Then, in the later stage, initialized by the neural predictions, we perform PBIR to refine the initial results and obtain the final high-quality reconstruction. Experimental results demonstrate our pipeline significantly outperforms existing reconstruction methods quality-wise and performance-wise.

翻译：基于二维图像（如照片）重建物理世界中物体的几何形状、空间变化表面外观及其周围光照，一直是计算机视觉与图形学领域的长期难题。本文提出了一种融合神经物体重建与物理逆渲染（PBIR）的鲁棒物体重建流程。具体而言，该流程首先利用神经阶段生成物体形状、反射率及光照的高质量但可能不完美的预测结果；随后，在后期阶段以神经预测结果为初始化，执行物理逆渲染对初始结果进行精细化处理，最终获得高保真重建。实验结果表明，本流程在重建质量与性能上均显著优于现有方法。