Efficiently generating a free-style 3D portrait with high quality and consistency is a promising yet challenging task. The portrait styles generated by most existing methods are usually restricted by their 3D generators, which are learned in specific facial datasets, such as FFHQ. To get a free-style 3D portrait, one can build a large-scale multi-style database to retrain the 3D generator, or use a off-the-shelf tool to do the style translation. However, the former is time-consuming due to data collection and training process, the latter may destroy the multi-view consistency. To tackle this problem, we propose a fast 3D portrait synthesis framework in this paper, which enable one to use text prompts to specify styles. Specifically, for a given portrait style, we first leverage two generative priors, a 3D-aware GAN generator (EG3D) and a text-guided image editor (Ip2p), to quickly construct a few-shot training set, where the inference process of Ip2p is optimized to make editing more stable. Then we replace original triplane generator of EG3D with a Image-to-Triplane (I2T) module for two purposes: 1) getting rid of the style constraints of pre-trained EG3D by fine-tuning I2T on the few-shot dataset; 2) improving training efficiency by fixing all parts of EG3D except I2T. Furthermore, we construct a multi-style and multi-identity 3D portrait database to demonstrate the scalability and generalization of our method. Experimental results show that our method is capable of synthesizing high-quality 3D portraits with specified styles in a few minutes, outperforming the state-of-the-art.

翻译：高效生成高质量且一致的自由风格三维肖像是一项前景广阔但充满挑战的任务。现有方法生成的肖像风格通常受限于其三维生成器在特定人脸数据集（如FFHQ）上的学习结果。为获得自由风格的三维肖像，一种方案是构建大规模多风格数据集重新训练三维生成器，另一种方案是使用现成工具进行风格迁移。然而前者因数据采集和训练过程耗时巨大，后者则可能破坏多视角一致性。针对这一问题，本文提出一种快速三维肖像合成框架，支持通过文本提示指定风格。具体而言，对于给定肖像风格，我们首先利用两种生成先验——三维感知的GAN生成器（EG3D）和文本引导的图像编辑器（Ip2p），快速构建小样本训练集，并优化Ip2p的推理过程以提升编辑稳定性。随后，我们将EG3D的原始三平面生成器替换为图像到三平面（I2T）模块，以实现双重目标：1）通过对I2T在小样本数据集上进行微调，消除预训练EG3D的风格约束；2）除I2T外冻结EG3D其余部分，提升训练效率。此外，我们构建了多风格多身份的三维肖像数据库，以验证方法的可扩展性与泛化能力。实验结果表明，本方法能在几分钟内合成高质量、指定风格的三维肖像，性能超越现有最优方法。