Face swapping has gained significant attention for its varied applications. The majority of previous face swapping approaches have relied on the seesaw game training scheme, which often leads to the instability of the model training and results in undesired samples with blended identities due to the target identity leakage problem. This paper introduces the Shape Agnostic Masked AutoEncoder (SAMAE) training scheme, a novel self-supervised approach designed to enhance face swapping model training. Our training scheme addresses the limitations of traditional training methods by circumventing the conventional seesaw game and introducing clear ground truth through its self-reconstruction training regime. It effectively mitigates identity leakage by masking facial regions of the input images and utilizing learned disentangled identity and non-identity features. Additionally, we tackle the shape misalignment problem with new techniques including perforation confusion and random mesh scaling, and establishes a new state-of-the-art, surpassing other baseline methods, preserving both identity and non-identity attributes, without sacrificing on either aspect.
翻译:人脸交换技术因其多种应用场景而备受关注。以往大多数换脸方法依赖于跷跷板博弈训练方案,这常导致模型训练不稳定,并因目标身份泄露问题产生身份特征混合的异常样本。本文提出形状无关掩码自编码器(SAMAE)训练方案——一种全新的自监督方法,旨在优化换脸模型训练。该训练方案通过规避传统跷跷板博弈、引入基于自重构训练范式的清晰真值,有效解决了传统训练方法的局限性。它通过掩码输入图像的面部区域,利用学习到的解耦身份特征与非身份特征,显著减轻了身份泄露问题。此外,我们通过穿孔混淆和随机网格缩放等新技术解决了形状错位难题,在不牺牲身份与非身份属性任何一方的前提下,超越了现有基线方法,达到了新的最优性能水平。