Lighting effects such as shadows or reflections are key in making synthetic images realistic and visually appealing. To generate such effects, traditional computer graphics uses a physically-based renderer along with 3D geometry. To compensate for the lack of geometry in 2D Image compositing, recent deep learning-based approaches introduced a pixel height representation to generate soft shadows and reflections. However, the lack of geometry limits the quality of the generated soft shadows and constrain reflections to pure specular ones. We introduce PixHt-Lab, a system leveraging an explicit mapping from pixel height representation to 3D space. Using this mapping, PixHt-Lab reconstructs both the cutout and background geometry and renders realistic, diverse, lighting effects for image compositing. Given a surface with physically-based materials, we can render reflections with varying glossiness. To generate more realistic soft shadows, we further propose to use 3D-aware buffer channels to guide a neural renderer. Both quantitative and qualitative evaluations demonstrate that PixHt-Lab significantly improves soft shadow generation.

翻译：阴影、倒影等光照效果是使合成图像逼真且具有视觉吸引力的关键。传统计算机图形学通过基于物理的渲染器结合三维几何生成此类效果。为弥补二维图像合成中缺少几何信息的不足，近期基于深度学习的方案引入像素高度表征以生成软阴影与倒影。然而，几何信息的匮乏限制了生成软阴影的质量，并使倒影局限于纯镜面反射。本文提出PixHt-Lab系统，利用像素高度表征到三维空间的显式映射。借助此映射，PixHt-Lab重构前景剪影与背景几何，为图像合成渲染出逼真且多样化的光照效果。对于具有基于物理材质的表面，系统可渲染不同光泽度的倒影。为生成更真实的软阴影，我们进一步提出利用三维感知缓冲通道引导神经渲染器。定量与定性评估均表明，PixHt-Lab显著提升了软阴影生成质量。