We present PIXLRelight, a feed-forward approach for physically controllable single-image relighting. Existing methods either provide limited lighting control (e.g. through text or environment maps), accumulate errors when chaining inverse and forward rendering, or require costly per-image optimization. Our key idea is to bridge physically based rendering (PBR) and learned image synthesis through a shared intrinsic conditioning that can be obtained from either real photographs or PBR renders. At training time, paired multi-illumination photographs are decomposed into albedo, diffuse shading, and non-diffuse residuals, which condition the model. At inference time, the same conditioning is computed from a path-traced render of a coarse 3D reconstruction of the input under user-specified PBR lights. A transformer-based neural renderer then applies the target illumination to the source photograph, preserving fine image detail through a per-pixel affine modulation. PIXLRelight enables arbitrary PBR-style lighting control, achieves state-of-the-art relighting quality, and runs in under a tenth of a second per image. Code and models are available at https://mlfarinha.github.io/pixl-relight/.
翻译:我们提出PIXLRelight,一种用于单张图像物理可控重光照的前馈方法。现有方法要么提供有限的光照控制(例如通过文本或环境贴图),要么在链式执行正向与逆向渲染时累积误差,要么需要代价高昂的逐图像优化。核心思路在于:通过共享的内在条件约束(可从真实照片或基于物理渲染的渲染结果中获取),将基于物理的渲染与学习型图像合成相衔接。训练阶段,配对的多光照照片被分解为反照率、漫反射着色项和非漫反射残差,以此约束模型;推理阶段,对输入图像的粗粒度三维重建结果施加用户指定的PBR光源,通过路径追踪渲染计算相同的条件约束。随后,基于Transformer的神经渲染器通过逐像素仿射调制保留精细图像细节,将目标光照应用于原始照片。PIXLRelight支持任意PBR风格的光照控制,实现了当前最优的重光照质量,且每张图像的处理时间不足0.1秒。代码与模型开源地址:https://mlfarinha.github.io/pixl-relight/。