We propose a novel method, StyLitGAN, for relighting and resurfacing generated images in the absence of labeled data. Our approach generates images with realistic lighting effects, including cast shadows, soft shadows, inter-reflections, and glossy effects, without the need for paired or CGI data. StyLitGAN uses an intrinsic image method to decompose an image, followed by a search of the latent space of a pre-trained StyleGAN to identify a set of directions. By prompting the model to fix one component (e.g., albedo) and vary another (e.g., shading), we generate relighted images by adding the identified directions to the latent style codes. Quantitative metrics of change in albedo and lighting diversity allow us to choose effective directions using a forward selection process. Qualitative evaluation confirms the effectiveness of our method.

翻译：我们提出了一种新颖方法StyLitGAN，用于在无标注数据条件下对生成图像进行重光照与表面重构。该方法无需配对数据或CGI数据即可生成具有真实感光照效果的图像，包括投射阴影、软阴影、相互反射及光泽效果。StyLitGAN采用内蕴图像方法分解图像，随后在预训练StyleGAN的隐空间中搜索一组方向向量。通过引导模型固定某一分量（如反照率）并变化另一分量（如明暗度），我们将识别出的方向向量叠加至隐式风格编码以生成重光照图像。基于反照率变化与光照多样性的定量指标，我们通过前向选择过程筛选有效方向。定性评估证实了本方法的有效性。