We address the problem of facial expression editing by controling the relative variation of facial action-unit (AU) from the same person. This enables us to edit this specific person's expression in a fine-grained, continuous and interpretable manner, while preserving their identity, pose, background and detailed facial attributes. Key to our model, which we dub MagicFace, is a diffusion model conditioned on AU variations and an ID encoder to preserve facial details of high consistency. Specifically, to preserve the facial details with the input identity, we leverage the power of pretrained Stable-Diffusion models and design an ID encoder to merge appearance features through self-attention. To keep background and pose consistency, we introduce an efficient Attribute Controller by explicitly informing the model of current background and pose of the target. By injecting AU variations into a denoising UNet, our model can animate arbitrary identities with various AU combinations, yielding superior results in high-fidelity expression editing compared to other facial expression editing works. Code is publicly available at https://github.com/weimengting/MagicFace.

翻译：本文通过控制同一人物面部动作单元（AU）的相对变化，解决了面部表情编辑问题。该方法能够以细粒度、连续且可解释的方式编辑特定人物的表情，同时保持其身份、姿态、背景及精细面部属性不变。我们提出的模型MagicFace的核心是一个以AU变化为条件的扩散模型，以及一个用于保持高一致性面部细节的身份编码器。具体而言，为保留输入身份的面部细节，我们利用预训练的Stable-Diffusion模型的能力，设计了一个通过自注意力机制融合外观特征的身份编码器。为保持背景与姿态一致性，我们引入了一个高效的属性控制器，通过显式告知模型当前目标的背景与姿态信息。通过将AU变化注入去噪UNet，我们的模型能够用多种AU组合驱动任意身份，在与其他面部表情编辑工作的对比中，实现了高保真表情编辑的优异效果。代码公开于https://github.com/weimengting/MagicFace。