We present a physics-based inverse rendering method that learns the illumination, geometry, and materials of a scene from posed multi-view RGB images. To model the illumination of a scene, existing inverse rendering works either completely ignore the indirect illumination or model it by coarse approximations, leading to sub-optimal illumination, geometry, and material prediction of the scene. In this work, we propose a physics-based illumination model that first locates surface points through an efficient refined sphere tracing algorithm, then explicitly traces the incoming indirect lights at each surface point based on reflection. Then, we estimate each identified indirect light through an efficient neural network. Moreover, we utilize the Leibniz's integral rule to resolve non-differentiability in the proposed illumination model caused by boundary lights inspired by differentiable irradiance in computer graphics. As a result, the proposed differentiable illumination model can be learned end-to-end together with geometry and materials estimation. As a side product, our physics-based inverse rendering model also facilitates flexible and realistic material editing as well as relighting. Extensive experiments on synthetic and real-world datasets demonstrate that the proposed method performs favorably against existing inverse rendering methods on novel view synthesis and inverse rendering.

翻译：我们提出了一种基于物理的反向渲染方法，该方法从多视角RGB图像中学习场景的光照、几何结构和材质。现有反向渲染工作在建模场景光照时，要么完全忽略间接光照，要么通过粗略近似进行建模，导致对场景光照、几何和材质的预测不够理想。本文提出了一种基于物理的光照模型，首先通过高效的优化球体追踪算法定位表面点，然后基于反射显式追踪每个表面点接收的间接光线，并通过高效的神经网络估计每个被识别的间接光照。此外，我们利用莱布尼茨积分法则解决了所提出的光照模型中由边界光线导致的不可微性问题，该思路受计算机图形学中可微分辐照度的启发。最终，所提出的可微分光照模型可与几何和材质估计一起进行端到端学习。作为副产品，我们的基于物理的反向渲染模型还支持灵活逼真的材质编辑和重新光照。在合成和真实数据集上的大量实验表明，所提方法在新视图合成和反向渲染方面优于现有方法。