Facial parts swapping aims to selectively transfer regions of interest from the source image onto the target image while maintaining the rest of the target image unchanged. Most studies on face swapping designed specifically for full-face swapping, are either unable or significantly limited when it comes to swapping individual facial parts, which hinders fine-grained and customized character designs. However, designing such an approach specifically for facial parts swapping is challenged by a reasonable multiple reference feature fusion, which needs to be both efficient and effective. To overcome this challenge, FuseAnyPart is proposed to facilitate the seamless "fuse-any-part" customization of the face. In FuseAnyPart, facial parts from different people are assembled into a complete face in latent space within the Mask-based Fusion Module. Subsequently, the consolidated feature is dispatched to the Addition-based Injection Module for fusion within the UNet of the diffusion model to create novel characters. Extensive experiments qualitatively and quantitatively validate the superiority and robustness of FuseAnyPart. Source codes are available at https://github.com/Thomas-wyh/FuseAnyPart.
翻译:面部局部替换旨在将源图像中感兴趣的区域选择性地迁移到目标图像上,同时保持目标图像的其余部分不变。大多数专门为全脸替换设计的面部替换研究,在替换单个面部局部时要么无法实现,要么受到显著限制,这阻碍了细粒度和定制化的角色设计。然而,专门为面部局部替换设计此类方法,面临着合理且高效有效的多参考特征融合的挑战。为克服这一挑战,本文提出了FuseAnyPart,以促进面部实现无缝的"任意局部融合"定制。在FuseAnyPart中,来自不同人物的面部局部在基于掩码的融合模块中于潜在空间内被组装成一张完整的面部。随后,整合后的特征被分发到基于加法的注入模块中,以便在扩散模型的UNet内部进行融合,从而创造新的角色。大量的定性和定量实验验证了FuseAnyPart的优越性和鲁棒性。源代码可在 https://github.com/Thomas-wyh/FuseAnyPart 获取。