In recent years, generative 3D face models (e.g., EG3D) have been developed to tackle the problem of synthesizing photo-realistic faces. However, these models are often unable to capture facial features unique to each individual, highlighting the importance of personalization. Some prior works have shown promise in personalizing generative face models, but these studies primarily focus on 2D settings. Also, these methods require both fine-tuning and storing a large number of parameters for each user, posing a hindrance to achieving scalable personalization. Another challenge of personalization is the limited number of training images available for each individual, which often leads to overfitting when using full fine-tuning methods. Our proposed approach, My3DGen, generates a personalized 3D prior of an individual using as few as 50 training images. My3DGen allows for novel view synthesis, semantic editing of a given face (e.g. adding a smile), and synthesizing novel appearances, all while preserving the original person's identity. We decouple the 3D facial features into global features and personalized features by freezing the pre-trained EG3D and training additional personalized weights through low-rank decomposition. As a result, My3DGen introduces only $\textbf{240K}$ personalized parameters per individual, leading to a $\textbf{127}\times$ reduction in trainable parameters compared to the $\textbf{30.6M}$ required for fine-tuning the entire parameter space. Despite this significant reduction in storage, our model preserves identity features without compromising the quality of downstream applications.

翻译：近年来，生成式三维人脸模型（如EG3D）已被开发用于解决合成逼真人脸的问题。然而，这些模型往往难以捕捉每个个体独特的面部特征，凸显了个性化的重要性。部分先前研究已在个性化生成人脸模型方面展现出潜力，但这些工作主要集中于二维场景。此外，这些方法需要为每位用户微调并存储大量参数，阻碍了可扩展个性化的实现。个性化的另一挑战是每位个体可用的训练图像数量有限，使用全参数微调方法时容易导致过拟合。我们提出的方法My3DGen，仅需50张训练图像即可生成个体的个性化三维先验。My3DGen支持新视角合成、对给定人脸进行语义编辑（例如添加微笑），以及合成新外观，同时保持原有身份特征。我们通过冻结预训练的EG3D，并利用低秩分解训练额外的个性化权重，将三维人脸特征解耦为全局特征与个性化特征。由此，My3DGen为每位个体仅引入$\textbf{240K}$个性化参数，相较于全参微调所需的$\textbf{30.6M}$参数，可训练参数减少$\textbf{127}$倍。尽管存储量显著降低，本模型仍能在不牺牲下游应用质量的前提下保留身份特征。