Our paper presents My3DGen, a practical system for creating a personalized and lightweight 3D generative prior using as few as 10 images. My3DGen can reconstruct multi-view consistent images from an input test image, and generate novel appearances by interpolating between any two images of the same individual. While recent studies have demonstrated the effectiveness of personalized generative priors in producing high-quality 2D portrait reconstructions and syntheses, to the best of our knowledge, we are the first to develop a personalized 3D generative prior. Instead of fine-tuning a large pre-trained generative model with millions of parameters to achieve personalization, we propose a parameter-efficient approach. Our method involves utilizing a pre-trained model with fixed weights as a generic prior, while training a separate personalized prior through low-rank decomposition of the weights in each convolution and fully connected layer. However, parameter-efficient few-shot fine-tuning on its own often leads to overfitting. To address this, we introduce a regularization technique based on symmetry of human faces. This regularization enforces that novel view renderings of a training sample, rendered from symmetric poses, exhibit the same identity. By incorporating this symmetry prior, we enhance the quality of reconstruction and synthesis, particularly for non-frontal (profile) faces. Our final system combines low-rank fine-tuning with symmetry regularization and significantly surpasses the performance of pre-trained models, e.g. EG3D. It introduces only approximately 0.6 million additional parameters per identity compared to 31 million for full finetuning of the original model. As a result, our system achieves a 50-fold reduction in model size without sacrificing the quality of the generated 3D faces. Code will be available at our project page: https://luchaoqi.github.io/my3dgen.

翻译：本文提出My3DGen，一个利用仅10张图像即可创建个性化轻量级三维生成先验的实用系统。My3DGen能够从输入测试图像重建多视角一致图像，并通过在同一主体的任意两张图像之间插值生成新外观。尽管近期研究已证明个性化生成先验在生成高质量二维肖像重建与合成方面的有效性，但据我们所知，我们是首个开发个性化三维生成先验的工作。我们未对具有数百万参数的大型预训练生成模型进行微调以实现个性化，而是提出了一种参数高效的方法。该方法利用固定权重的预训练模型作为通用先验，同时通过每个卷积层和全连接层权重的低秩分解来训练独立的个性化先验。然而，单纯的参数高效少样本微调往往会导致过拟合。为解决此问题，我们引入了一种基于人脸对称性的正则化技术。该正则化强制训练样本在对称姿态下渲染的新视角视图具有相同身份。通过融入这一对称先验，我们提升了重建与合成的质量，尤其对于非正面（侧面）人脸。最终系统将低秩微调与对称正则化相结合，显著超越了预训练模型（例如EG3D）的性能。每个身份仅引入约0.6百万额外参数，而原模型完全微调需3100万参数。因此，我们的系统在不牺牲生成三维人脸质量的前提下，实现了50倍的模型尺寸缩减。代码将发布于项目页面：https://luchaoqi.github.io/my3dgen。