A 3D digital scene contains many components: lights, materials and geometries, interacting to reach the desired appearance. Staging such a scene is time-consuming and requires both artistic and technical skills. In this work, we propose PSDR-Room, a system allowing to optimize lighting as well as the pose and materials of individual objects to match a target image of a room scene, with minimal user input. To this end, we leverage a recent path-space differentiable rendering approach that provides unbiased gradients of the rendering with respect to geometry, lighting, and procedural materials, allowing us to optimize all of these components using gradient descent to visually match the input photo appearance. We use recent single-image scene understanding methods to initialize the optimization and search for appropriate 3D models and materials. We evaluate our method on real photographs of indoor scenes and demonstrate the editability of the resulting scene components.

翻译：三维数字场景包含多个相互作用的组件：光照、材质与几何体，它们共同呈现最终视觉效果。构建此类场景不仅耗时，还需要兼具艺术素养与技术能力。本文提出PSDR-Room系统，该系统能够在最小人工干预下，通过优化光照、个体物体的姿态及材质，使房间场景与目标图像相匹配。为此，我们采用最新的路径空间可微渲染方法，该方法可提供关于几何体、光照与程序化材质的无偏渲染梯度，使我们能够通过梯度下降优化所有组件以在视觉上匹配输入照片。我们利用近期单图像场景理解方法初始化优化过程，并自动搜索合适的3D模型与材质。通过室内场景真实照片的评估，我们验证了该方法在保持场景组件可编辑性方面的有效性。