A 3D avatar typically has one of six cardinal facial expressions. To simulate realistic emotional variation, we should be able to render a facial transition between two arbitrary expressions. This study presents a new framework for instruction-driven facial expression generation that produces a 3D face and, starting from an image of the face, transforms the facial expression from one designated facial expression to another. The Instruction-driven Facial Expression Decomposer (IFED) module is introduced to facilitate multimodal data learning and capture the correlation between textual descriptions and facial expression features. Subsequently, we propose the Instruction to Facial Expression Transition (I2FET) method, which leverages IFED and a vertex reconstruction loss function to refine the semantic comprehension of latent vectors, thus generating a facial expression sequence according to the given instruction. Lastly, we present the Facial Expression Transition model to generate smooth transitions between facial expressions. Extensive evaluation suggests that the proposed model outperforms state-of-the-art methods on the CK+ and CelebV-HQ datasets. The results show that our framework can generate facial expression trajectories according to text instruction. Considering that text prompts allow us to make diverse descriptions of human emotional states, the repertoire of facial expressions and the transitions between them can be expanded greatly. We expect our framework to find various practical applications More information about our project can be found at https://vohoanganh.github.io/tg3dfet/

翻译：三维虚拟形象通常具有六种基本面部表情之一。为模拟真实的情感变化，我们应能渲染任意两种表情之间的面部过渡。本研究提出了一种新的指令驱动面部表情生成框架，该框架可生成三维人脸，并从人脸图像出发，将面部表情从一种指定表情转换为另一种。我们引入了指令驱动面部表情分解器模块，以促进多模态数据学习并捕获文本描述与面部表情特征之间的关联。随后，我们提出了指令到面部表情过渡方法，该方法利用IFED模块和顶点重建损失函数来优化潜在向量的语义理解，从而根据给定指令生成面部表情序列。最后，我们提出了面部表情过渡模型以生成表情间的平滑过渡。大量评估表明，所提模型在CK+和CelebV-HQ数据集上优于现有先进方法。结果表明，我们的框架能够根据文本指令生成面部表情轨迹。考虑到文本提示允许我们对人类情感状态进行多样化描述，面部表情库及其间的过渡可得到极大扩展。我们预期该框架将在多种实际应用中找到用武之地。更多项目信息请访问：https://vohoanganh.github.io/tg3dfet/