We present a learning-based approach to relight a single image of Lambertian and low-frequency specular objects. Our method enables inserting objects from photographs into new scenes and relighting them under the new environment lighting, which is essential for AR applications. To relight the object, we solve both inverse rendering and re-rendering. To resolve the ill-posed inverse rendering, we propose a weakly-supervised method by a low-rank constraint. To facilitate the weakly-supervised training, we contribute Relit, a large-scale (750K images) dataset of videos with aligned objects under changing illuminations. For re-rendering, we propose a differentiable specular rendering layer to render low-frequency non-Lambertian materials under various illuminations of spherical harmonics. The whole pipeline is end-to-end and efficient, allowing for a mobile app implementation of AR object insertion. Extensive evaluations demonstrate that our method achieves state-of-the-art performance. Project page: https://renjiaoyi.github.io/relighting/.

翻译：我们提出一种基于学习的方法，用于对朗伯体和低频镜面反射物体的单张图像进行重光照。该方法可将照片中的物体插入新场景，并在新环境光照下对其重光照，这对增强现实应用至关重要。为解决物体的重光照问题，我们同时处理逆渲染和重渲染。针对逆渲染的病态性，我们提出一种基于低秩约束的弱监督方法。为支持弱监督训练，我们构建了Relit数据集——一个包含75万张图像的大规模视频数据集，其中物体在变化光照下保持对齐。在重渲染方面，我们提出一种可微分镜面反射层，用于在球谐函数表示的各种光照下渲染低频非朗伯材质。整个流程端到端且高效，可支持移动端增强现实物体插入的实现。大量评估表明，我们的方法达到了业界领先水平。项目主页：https://renjiaoyi.github.io/relighting/。