In recent years, the field of talking faces generation has attracted considerable attention, with certain methods adept at generating virtual faces that convincingly imitate human expressions. However, existing methods face challenges related to limited generalization, particularly when dealing with challenging identities. Furthermore, methods for editing expressions are often confined to a singular emotion, failing to adapt to intricate emotions. To overcome these challenges, this paper proposes EmoTalker, an emotionally editable portraits animation approach based on the diffusion model. EmoTalker modifies the denoising process to ensure preservation of the original portrait's identity during inference. To enhance emotion comprehension from text input, Emotion Intensity Block is introduced to analyze fine-grained emotions and strengths derived from prompts. Additionally, a crafted dataset is harnessed to enhance emotion comprehension within prompts. Experiments show the effectiveness of EmoTalker in generating high-quality, emotionally customizable facial expressions.

翻译：近年来，说话人脸生成领域引起了广泛关注，某些方法能够生成逼真模仿人类表情的虚拟人脸。然而，现有方法面临泛化能力有限的挑战，尤其在处理具有挑战性的身份特征时表现欠佳。此外，表情编辑方法通常局限于单一情感，无法适应复杂多变的情感需求。为克服这些难题，本文提出EmoTalker——一种基于扩散模型的可编辑情感肖像动画方法。EmoTalker通过修改去噪过程，确保在推理时保留原始肖像的身份特征。为提升对文本输入的情感理解能力，引入情感强度模块（Emotion Intensity Block）以解析提示词中的细粒度情感及强度。此外，利用精心构建的增强数据集来提升提示词中的情感理解能力。实验证明，EmoTalker在生成高质量、可定制情感的面部表情方面具有显著效果。