The goal of inverse rendering is to decompose geometry, lights, and materials given pose multi-view images. To achieve this goal, we propose neural direct and joint inverse rendering, NDJIR. Different from prior works which relies on some approximations of the rendering equation, NDJIR directly addresses the integrals in the rendering equation and jointly decomposes geometry: signed distance function, lights: environment and implicit lights, materials: base color, roughness, specular reflectance using the powerful and flexible volume rendering framework, voxel grid feature, and Bayesian prior. Our method directly uses the physically-based rendering, so we can seamlessly export an extracted mesh with materials to DCC tools and show material conversion examples. We perform intensive experiments to show that our proposed method can decompose semantically well for real object in photogrammetric setting and what factors contribute towards accurate inverse rendering.

翻译：逆渲染的目标是在给定多视角姿态图像的情况下分解几何、光照和材质。为了实现这一目标，我们提出了神经直接联合逆渲染方法NDJIR。与依赖渲染方程近似求解的现有工作不同，NDJIR直接处理渲染方程中的积分项，并利用强大且灵活的体渲染框架、体素网格特征以及贝叶斯先验，联合分解几何（符号距离函数）、光照（环境光与隐式光）以及材质（基色、粗糙度、高光反射率）。我们的方法直接基于物理渲染，因此能够将提取的网格及其材质无缝导出至DCC工具，并展示材质转换示例。通过大量实验，我们证明了所提方法在摄影测量场景下能够对真实物体进行语义级良好的分解，并揭示了影响精确逆渲染的关键因素。