Generating photorealistic 3D faces from given conditions is a challenging task. Existing methods often rely on time-consuming one-by-one optimization approaches, which are not efficient for modeling the same distribution content, e.g., faces. Additionally, an ideal controllable 3D face generation model should consider both facial attributes and expressions. Thus we propose a novel approach called TEx-Face(TExt & Expression-to-Face) that addresses these challenges by dividing the task into three components, i.e., 3D GAN Inversion, Conditional Style Code Diffusion, and 3D Face Decoding. For 3D GAN inversion, we introduce two methods which aim to enhance the representation of style codes and alleviate 3D inconsistencies. Furthermore, we design a style code denoiser to incorporate multiple conditions into the style code and propose a data augmentation strategy to address the issue of insufficient paired visual-language data. Extensive experiments conducted on FFHQ, CelebA-HQ, and CelebA-Dialog demonstrate the promising performance of our TEx-Face in achieving the efficient and controllable generation of photorealistic 3D faces. The code will be available at https://github.com/sxl142/TEx-Face.

翻译：从给定条件生成逼真三维人脸是一项具有挑战性的任务。现有方法通常依赖耗时的逐样本优化方式，难以高效建模具有相同分布特征的内容（例如人脸）。此外，理想的可控三维人脸生成模型需同时兼顾面部属性与表情。为此，我们提出名为TEx-Face（文本与表情到人脸）的创新方法，通过将任务分解为三维生成对抗网络反演、条件式风格编码扩散和三维人脸解码三个模块来解决上述挑战。在三维GAN反演方面，我们引入两种方法以增强风格编码表征能力并缓解三维不一致性问题。进一步地，我们设计风格编码去噪器将多模态条件融入风格编码，并提出数据增强策略解决视觉-语言配对数据不足的问题。在FFHQ、CelebA-HQ和CelebA-Dialog数据集上开展的大量实验表明，TEx-Face在实现高效可控的逼真三维人脸生成方面展现出卓越性能。相关代码将开源至https://github.com/sxl142/TEx-Face。