Current face reenactment and swapping methods mainly rely on GAN frameworks, but recent focus has shifted to pre-trained diffusion models for their superior generation capabilities. However, training these models is resource-intensive, and the results have not yet achieved satisfactory performance levels. To address this issue, we introduce Face-Adapter, an efficient and effective adapter designed for high-precision and high-fidelity face editing for pre-trained diffusion models. We observe that both face reenactment/swapping tasks essentially involve combinations of target structure, ID and attribute. We aim to sufficiently decouple the control of these factors to achieve both tasks in one model. Specifically, our method contains: 1) A Spatial Condition Generator that provides precise landmarks and background; 2) A Plug-and-play Identity Encoder that transfers face embeddings to the text space by a transformer decoder. 3) An Attribute Controller that integrates spatial conditions and detailed attributes. Face-Adapter achieves comparable or even superior performance in terms of motion control precision, ID retention capability, and generation quality compared to fully fine-tuned face reenactment/swapping models. Additionally, Face-Adapter seamlessly integrates with various StableDiffusion models.

翻译：当前的面部重演与交换方法主要依赖生成对抗网络框架，但近期研究焦点已转向预训练扩散模型，因其具备更优的生成能力。然而，训练此类模型需要大量计算资源，且已有结果尚未达到令人满意的性能水平。为解决该问题，我们提出Face-Adapter——一种专为预训练扩散模型设计的高效适配器，可实现高精度、高保真度的面部编辑。观察到面部重演与交换任务本质上均涉及目标结构、身份与属性的组合，我们旨在充分解耦这些因素的控制，从而在单一模型中同时实现两类任务。具体而言，本方法包含：1）空间条件生成器，用于提供精确的特征点与背景信息；2）即插即用的身份编码器，通过Transformer解码器将面部嵌入映射至文本空间；3）属性控制器，用于整合空间条件与细粒度属性信息。与完全微调的面部重演/交换模型相比，Face-Adapter在运动控制精度、身份保持能力及生成质量方面达到甚至超越现有性能水平。此外，Face-Adapter可无缝集成多种StableDiffusion模型。