We propose a generative framework, FaceLit, capable of generating a 3D face that can be rendered at various user-defined lighting conditions and views, learned purely from 2D images in-the-wild without any manual annotation. Unlike existing works that require careful capture setup or human labor, we rely on off-the-shelf pose and illumination estimators. With these estimates, we incorporate the Phong reflectance model in the neural volume rendering framework. Our model learns to generate shape and material properties of a face such that, when rendered according to the natural statistics of pose and illumination, produces photorealistic face images with multiview 3D and illumination consistency. Our method enables photorealistic generation of faces with explicit illumination and view controls on multiple datasets - FFHQ, MetFaces and CelebA-HQ. We show state-of-the-art photorealism among 3D aware GANs on FFHQ dataset achieving an FID score of 3.5.

翻译：我们提出一个生成框架FaceLit，能够生成可在用户自定义光照条件和视角下渲染的三维人脸，该框架仅从野外二维图像中学习，无需任何人工标注。与现有需要精细采集设置或人工劳动的工作不同，我们依赖于现成的姿态和光照估计器。利用这些估计值，我们将Phong反射模型融入神经体积渲染框架中。我们的模型学习生成人脸的形状和材质属性，使得根据姿态和光照的自然统计特性进行渲染时，能够产生具有多视角三维和光照一致性的逼真人脸图像。该方法能够在多个数据集（FFHQ、MetFaces和CelebA-HQ）上实现具有显式光照和视角控制的逼真人脸生成。我们在FFHQ数据集上展示了三维感知GAN中的最先进逼真度，FID分数达到3.5。