Face inpainting aims at plausibly predicting missing pixels of face images within a corrupted region. Most existing methods rely on generative models learning a face image distribution from a big dataset, which produces uncontrollable results, especially with large-scale missing regions. To introduce strong control for face inpainting, we propose a novel reference-guided face inpainting method that fills the large-scale missing region with identity and texture control guided by a reference face image. However, generating high-quality results under imposing two control signals is challenging. To tackle such difficulty, we propose a dual control one-stage framework that decouples the reference image into two levels for flexible control: High-level identity information and low-level texture information, where the identity information figures out the shape of the face and the texture information depicts the component-aware texture. To synthesize high-quality results, we design two novel modules referred to as Half-AdaIN and Component-Wise Style Injector (CWSI) to inject the two kinds of control information into the inpainting processing. Our method produces realistic results with identity and texture control faithful to reference images. To the best of our knowledge, it is the first work to concurrently apply identity and component-level controls in face inpainting to promise more precise and controllable results. Code is available at https://github.com/WuyangLuo/RefFaceInpainting

翻译：人脸修复旨在合理预测受损区域内人脸图像的缺失像素。现有方法多依赖生成模型从大数据集中学习人脸图像分布，这会导致结果难以控制，尤其在大面积缺失区域中。为引入对人脸修复的强控制，我们提出一种新颖的参考引导人脸修复方法，通过参考人脸图像引导，在身份与纹理控制下填充大面积缺失区域。然而，在施加两种控制信号时生成高质量结果具有挑战性。为应对这一难题，我们提出一种双控制单阶段框架，将参考图像解耦为两个层次以实现灵活控制：高层身份信息与低层纹理信息，其中身份信息确定面部形状，纹理信息描述组件感知纹理。为合成高质量结果，我们设计了两种新型模块——半自适应实例归一化（Half-AdaIN）与组件式风格注入器（CWSI），将两类控制信息注入修复过程。我们的方法能够生成忠于参考图像的身份与纹理控制逼真结果。据我们所知，这是首个在人脸修复中同时应用身份与组件级控制以实现更精确、可控结果的工作。代码开源在 https://github.com/WuyangLuo/RefFaceInpainting