Inverse rendering methods aim to estimate geometry, materials and illumination from multi-view RGB images. In order to achieve better decomposition, recent approaches attempt to model indirect illuminations reflected from different materials via Spherical Gaussians (SG), which, however, tends to blur the high-frequency reflection details. In this paper, we propose an end-to-end inverse rendering pipeline that decomposes materials and illumination from multi-view images, while considering near-field indirect illumination. In a nutshell, we introduce the Monte Carlo sampling based path tracing and cache the indirect illumination as neural radiance, enabling a physics-faithful and easy-to-optimize inverse rendering method. To enhance efficiency and practicality, we leverage SG to represent the smooth environment illuminations and apply importance sampling techniques. To supervise indirect illuminations from unobserved directions, we develop a novel radiance consistency constraint between implicit neural radiance and path tracing results of unobserved rays along with the joint optimization of materials and illuminations, thus significantly improving the decomposition performance. Extensive experiments demonstrate that our method outperforms the state-of-the-art on multiple synthetic and real datasets, especially in terms of inter-reflection decomposition.

翻译：逆渲染方法旨在从多视角RGB图像中估计几何、材质与光照。为实现更优的分解效果，现有方法尝试通过球面高斯函数建模不同材质反射的间接光照，但此类方法易模糊高频反射细节。本文提出一种端到端逆渲染管线，可在考虑近场间接光照的同时，从多视角图像分解材质与光照。简言之，我们引入基于蒙特卡洛采样的路径追踪技术，并将间接光照缓存为神经辐射场，从而构建物理保真且易于优化的逆渲染方法。为提升效率与实用性，我们利用球面高斯函数表达平滑环境光照并应用重要性采样技术。针对未观测方向的间接光照监督问题，我们提出隐式神经辐射场与未观测光线路径追踪结果之间的新型辐射一致性约束，通过联合优化材质与光照，显著提升分解性能。大量实验表明，本方法在多个合成与真实数据集上均优于现有技术，尤其在相互反射分解方面表现突出。