In this work, we focus on exploring explicit fine-grained control of generative facial image editing, all while generating faithful and consistent personalized facial appearances. We identify the key challenge of this task as the exploration of disentangled conditional control in the generation process, and accordingly propose a novel diffusion-based framework, named DisControlFace, comprising two decoupled components. Specifically, we leverage an off-the-shelf diffusion reconstruction model as the backbone and freeze its pre-trained weights, which helps to reduce identity shift and recover editing-unrelated details of the input image. Furthermore, we construct a parallel control network that is compatible with the reconstruction backbone to generate spatial control conditions based on estimated explicit face parameters. Finally, we further reformulate the training pipeline into a masked-autoencoding form to effectively achieve disentangled training of our DisControlFace. Our DisControlNet can perform robust editing on any facial image through training on large-scale 2D in-the-wild portraits and also supports low-cost fine-tuning with few additional images to further learn diverse personalized priors of a specific person. Extensive experiments demonstrate that DisControlFace can generate realistic facial images corresponding to various face control conditions, while significantly improving the preservation of the personalized facial details.

翻译：本文聚焦于探索生成式人脸图像编辑中的显式细粒度控制，同时生成忠实且一致的个性化人脸外观。我们确定该任务的关键挑战在于生成过程中对去纠缠条件控制的探索，并据此提出一种名为DisControlFace的新型扩散框架，该框架包含两个解耦组件。具体而言，我们利用现成的扩散重建模型作为主干网络并冻结其预训练权重，这有助于减少身份偏移并恢复输入图像中与编辑无关的细节。此外，我们构建了一个与重建主干网络兼容的并行控制网络，以基于估计出的显式人脸参数生成空间控制条件。最后，我们进一步将训练流程重构为掩码自编码形式，以有效实现DisControlFace的去纠缠训练。我们的DisControlNet可通过在大规模二维野外肖像数据上的训练，对任意人脸图像进行鲁棒的编辑，并支持通过少量额外图像进行低成本微调，以进一步学习特定人物的多样化个性化先验。大量实验表明，DisControlFace能够生成与各种人脸控制条件对应的逼真人脸图像，同时显著提升个性化人脸细节的保留能力。