Reconstructing the shape and spatially varying surface appearances of a physical-world object as well as its surrounding illumination based on 2D images (e.g., photographs) of the object has been a long-standing problem in computer vision and graphics. In this paper, we introduce a robust object reconstruction pipeline combining neural based object reconstruction and physics-based inverse rendering (PBIR). Specifically, our pipeline firstly leverages a neural stage to produce high-quality but potentially imperfect predictions of object shape, reflectance, and illumination. Then, in the later stage, initialized by the neural predictions, we perform PBIR to refine the initial results and obtain the final high-quality reconstruction. Experimental results demonstrate our pipeline significantly outperforms existing reconstruction methods quality-wise and performance-wise.

翻译：从物理世界物体的二维图像（例如照片）中重建其形状、空间变化表面外观及其周围光照，一直是计算机视觉与图形学领域的长期难题。本文提出了一种结合神经物体重建与基于物理的逆向渲染（PBIR）的鲁棒物体重建流程。具体而言，该流程首先利用神经阶段生成物体形状、反射率和光照的高质量但可能存在瑕疵的预测结果；随后在后续阶段中，以神经预测结果为初始化，执行PBIR对初始结果进行精化，最终获取高质量重建。实验结果表明，本流程在重建质量和性能上均显著优于现有方法。