Unsupervised learning of 3D human faces from unstructured 2D image data is an active research area. While recent works have achieved an impressive level of photorealism, they commonly lack control of lighting, which prevents the generated assets from being deployed in novel environments. To this end, we introduce LumiGAN, an unconditional Generative Adversarial Network (GAN) for 3D human faces with a physically based lighting module that enables relighting under novel illumination at inference time. Unlike prior work, LumiGAN can create realistic shadow effects using an efficient visibility formulation that is learned in a self-supervised manner. LumiGAN generates plausible physical properties for relightable faces, including surface normals, diffuse albedo, and specular tint without any ground truth data. In addition to relightability, we demonstrate significantly improved geometry generation compared to state-of-the-art non-relightable 3D GANs and notably better photorealism than existing relightable GANs.

翻译：从非结构化2D图像数据中无监督学习3D人脸是一个活跃的研究领域。尽管近期工作已实现令人惊叹的照片级真实感，但它们通常缺乏对光照的控制，导致生成的资产无法部署在新环境中。为此，我们提出LumiGAN——一种基于物理照明模块的无条件生成对抗网络（GAN），用于生成可在推理时在新光照下重照明的3D人脸。与先前工作不同，LumiGAN通过自监督方式学习的有效可见性公式，能够创建逼真的阴影效果。该网络无需任何真值数据，即可生成适用于可重照明人脸的合理物理属性，包括表面法线、漫反射反照率及镜面色调。除可重照明能力外，我们还证明了相比最先进的不可重照明3D GAN在几何生成质量上的显著提升，以及相比现有可重照明GAN在照片真实感方面的明显优势。