Reflectance bounds the frequency spectrum of illumination in the object appearance. In this paper, we introduce the first stochastic inverse rendering method, which recovers the full frequency spectrum of an illumination jointly with the object reflectance from a single image. Our key idea is to solve this blind inverse problem in the reflectance map, an appearance representation invariant to the underlying geometry, by learning to reverse the image formation with a novel diffusion model which we refer to as the Diffusion Reflectance Map Network (DRMNet). Given an observed reflectance map converted and completed from the single input image, DRMNet generates a reflectance map corresponding to a perfect mirror sphere while jointly estimating the reflectance. The forward process can be understood as gradually filtering a natural illumination with lower and lower frequency reflectance and additive Gaussian noise. DRMNet learns to invert this process with two subnetworks, IllNet and RefNet, which work in concert towards this joint estimation. The network is trained on an extensive synthetic dataset and is demonstrated to generalize to real images, showing state-of-the-art accuracy on established datasets.

翻译：反射率约束了物体外观中光照的频率谱。本文首次提出了一种随机逆渲染方法，该方法能够从单幅图像中联合恢复光照的完整频率谱和物体反射率。我们的核心思想是在反射率图中解决这一盲逆问题，反射率图是一种与底层几何结构无关的外观表示，通过训练一种新颖的扩散模型来逆转图像形成过程，我们将其称为扩散反射率图网络（DRMNet）。给定从单幅输入图像转换并补全的观测反射率图，DRMNet生成对应于完美镜面球的反射率图，同时联合估计反射率。前向过程可理解为利用频率逐渐降低的反射率函数和加性高斯噪声，逐步对自然光照进行滤波。DRMNet通过学习逆转这一过程，利用两个子网络IllNet和RefNet协同实现联合估计。该网络在大规模合成数据集上训练，并展现出对真实图像的泛化能力，在现有数据集上达到了最先进的精度。