Inverse rendering, the process of inferring scene properties from images, is a challenging inverse problem. The task is ill-posed, as many different scene configurations can give rise to the same image. Most existing solutions incorporate priors into the inverse-rendering pipeline to encourage plausible solutions, but they do not consider the inherent ambiguities and the multi-modal distribution of possible decompositions. In this work, we propose a novel scheme that integrates a denoising diffusion probabilistic model pre-trained on natural illumination maps into an optimization framework involving a differentiable path tracer. The proposed method allows sampling from combinations of illumination and spatially-varying surface materials that are, both, natural and explain the image observations. We further conduct an extensive comparative study of different priors on illumination used in previous work on inverse rendering. Our method excels in recovering materials and producing highly realistic and diverse environment map samples that faithfully explain the illumination of the input images.

翻译：逆向渲染是从图像推断场景属性的过程，是一个具有挑战性的逆向问题。由于多种不同的场景配置可能产生相同的图像，该任务本质上是病态的。现有的大多数解决方案将先验信息引入逆向渲染管线以鼓励合理的解，但未考虑固有歧义及可能分解的多模态分布。本文提出一种新颖方案，将预训练于自然光照图上的去噪扩散概率模型集成到包含可微分路径追踪器的优化框架中。该方法能够采样兼具自然性且符合图像观测的光照与空间变化表面材质组合。我们进一步对逆向渲染领域先前工作中使用的不同光照先验方法进行了广泛对比研究。本方法在材质恢复方面表现卓越，能生成高度逼真且多样化的环境图样本，精准解释输入图像的照明信息。