3D Gaussian Splatting (3DGS) has become a standard approach to reconstruct and render photorealistic 3D head avatars. A major challenge is to relight the avatars to match any scene illumination. For high quality relighting, existing methods require subjects to be captured under complex time-multiplexed illumination, such as one-light-at-a-time (OLAT). We propose a new generalized relightable 3D Gaussian head model that can relight any subject observed in a single- or multi-view images without requiring OLAT data for that subject. Our core idea is to learn a mapping from flat-lit 3DGS avatars to corresponding relightable Gaussian parameters for that avatar. Our model consists of two stages: a first stage that models flat-lit 3DGS avatars without OLAT lighting, and a second stage that learns the mapping to physically-based reflectance parameters for high-quality relighting. This two-stage design allows us to train the first stage across diverse existing multi-view datasets without OLAT lighting ensuring cross-subject generalization, where we learn a dataset-specific lighting code for self-supervised lighting alignment. Subsequently, the second stage can be trained on a significantly smaller dataset of subjects captured under OLAT illumination. Together, this allows our method to generalize well and relight any subject from the first stage as if we had captured them under OLAT lighting. Furthermore, we can fit our model to unseen subjects from as little as a single image, allowing several applications in novel view synthesis and relighting for digital avatars.

翻译：3D高斯泼溅（3DGS）已成为重建和渲染逼真3D头部化身的标准方法。一个主要挑战是如何对化身进行重光照以匹配任意场景照明。为实现高质量重光照，现有方法要求拍摄对象在复杂的时间复用照明条件下（如逐光源照明）被捕获。我们提出了一种新的通用可重光照3D高斯头部模型，能够对单视图或多视图图像中观察到的任意对象进行重光照，且无需该对象的逐光源照明数据。我们的核心思想是学习从平光照明3DGS化身到该化身对应可重光照高斯参数的映射。模型包含两个阶段：第一阶段建模无需逐光源照明的平光照明3DGS化身；第二阶段学习向基于物理的反射率参数的映射以实现高质量重光照。这种两阶段设计使我们能够在无需逐光源照明的多样化现有多视图数据集上训练第一阶段，确保跨对象泛化能力，并通过数据集特定的光照编码实现自监督光照对齐。随后，第二阶段可在显著更小的逐光源照明拍摄对象数据集上进行训练。这种设计使我们的方法能够良好泛化，并对第一阶段中的任意对象进行重光照，效果如同在逐光源照明条件下拍摄。此外，我们的模型仅需单张图像即可适配未见过的对象，为数字化身的新视角合成与重光照提供了多种应用可能。