We target cross-domain face reenactment in this paper, i.e., driving a cartoon image with the video of a real person and vice versa. Recently, many works have focused on one-shot talking face generation to drive a portrait with a real video, i.e., within-domain reenactment. Straightforwardly applying those methods to cross-domain animation will cause inaccurate expression transfer, blur effects, and even apparent artifacts due to the domain shift between cartoon and real faces. Only a few works attempt to settle cross-domain face reenactment. The most related work AnimeCeleb requires constructing a dataset with pose vector and cartoon image pairs by animating 3D characters, which makes it inapplicable anymore if no paired data is available. In this paper, we propose a novel method for cross-domain reenactment without paired data. Specifically, we propose a transformer-based framework to align the motions from different domains into a common latent space where motion transfer is conducted via latent code addition. Two domain-specific motion encoders and two learnable motion base memories are used to capture domain properties. A source query transformer and a driving one are exploited to project domain-specific motion to the canonical space. The edited motion is projected back to the domain of the source with a transformer. Moreover, since no paired data is provided, we propose a novel cross-domain training scheme using data from two domains with the designed analogy constraint. Besides, we contribute a cartoon dataset in Disney style. Extensive evaluations demonstrate the superiority of our method over competing methods.

翻译：本文针对跨领域面部重现问题，即利用真人视频驱动卡通图像，或反之亦然。近期诸多研究聚焦于单次说话人脸生成——利用真实视频驱动肖像，即领域内面部重现。然而，若将此类方法直接应用于跨领域动画，会因卡通与真实人脸间的领域偏移导致表情迁移不准确、模糊伪影甚至明显瑕疵。目前仅有少量研究尝试解决跨领域面部重现。最相关的工作AnimeCeleb通过构建3D角色动画化生成的姿态向量与卡通图像配对数据集，但若无配对数据则该方法不再适用。本文提出一种无需配对数据的新型跨领域重现方法。具体而言，我们构建基于Transformer的框架，将不同领域的运动对齐至共享隐空间，并通过隐编码加法实现运动迁移。采用两个领域专属运动编码器与两个可学习运动基记忆库捕捉领域特性，利用源查询Transformer与驱动查询Transformer将领域特定运动投影至规范空间，再通过Transformer将编辑后的运动反向投影至源领域。针对无配对数据问题，我们提出基于类比约束的双领域数据跨域训练方案。此外，我们构建了迪士尼风格卡通数据集。大量实验证明本方法优于现有竞争方法。