Generating high-fidelity 3D head avatars from a single image is challenging, as current methods lack fine-grained, intuitive control over expressions via text. This paper proposes SIE3D, a framework that generates expressive 3D avatars from a single image and descriptive text. SIE3D fuses identity features from the image with semantic embedding from text through a novel conditioning scheme, enabling detailed control. To ensure generated expressions accurately match the text, it introduces an innovative perceptual expression loss function. This loss uses a pre-trained expression classifier to regularize the generation process, guaranteeing expression accuracy. Extensive experiments show SIE3D significantly improves controllability and realism, outperforming competitive methods in identity preservation and expression fidelity on a single consumer-grade GPU. Project page: https://huang-zhiqi.github.io/SIE3D/
翻译:从单张图像生成高保真三维头部头像具有挑战性,现有方法难以通过文本实现精细、直观的表情控制。本文提出SIE3D框架,该框架能从单张图像和描述性文本生成可表达的三维头像。SIE3D通过新颖的条件调控方案,将图像的身份特征与文本的语义嵌入相融合,实现精细控制。为确保生成的表情与文本精确匹配,本文引入创新的感知表情损失函数——该损失利用预训练的表情分类器约束生成过程,保障表情准确性。大量实验表明,SIE3D在可控性和真实感方面显著提升,在单张消费级GPU上,其在身份保持和表情保真度方面均优于现有竞争方法。项目主页:https://huang-zhiqi.github.io/SIE3D/