Recent generative-prior-based methods have shown promising blind face restoration performance. They usually project the degraded images to the latent space and then decode high-quality faces either by single-stage latent optimization or directly from the encoding. Generating fine-grained facial details faithful to inputs remains a challenging problem. Most existing methods produce either overly smooth outputs or alter the identity as they attempt to balance between generation and reconstruction. This may be attributed to the typical trade-off between quality and resolution in the latent space. If the latent space is highly compressed, the decoded output is more robust to degradations but shows worse fidelity. On the other hand, a more flexible latent space can capture intricate facial details better, but is extremely difficult to optimize for highly degraded faces using existing techniques. To address these issues, we introduce a diffusion-based-prior inside a VQGAN architecture that focuses on learning the distribution over uncorrupted latent embeddings. With such knowledge, we iteratively recover the clean embedding conditioning on the degraded counterpart. Furthermore, to ensure the reverse diffusion trajectory does not deviate from the underlying identity, we train a separate Identity Recovery Network and use its output to constrain the reverse diffusion process. Specifically, using a learnable latent mask, we add gradients from a face-recognition network to a subset of latent features that correlates with the finer identity-related details in the pixel space, leaving the other features untouched. Disentanglement between perception and fidelity in the latent space allows us to achieve the best of both worlds. We perform extensive evaluations on multiple real and synthetic datasets to validate the superiority of our approach.

翻译：基于生成先验的方法在盲脸修复任务中已展现出良好效果。这类方法通常将退化图像投影至潜空间，随后通过单阶段潜空间优化或直接编码解码出高质量人脸。然而，生成与输入高度一致的细粒度面部细节仍具挑战性。现有方法为平衡生成质量与重建保真度，往往导致输出过度平滑或改变身份特征，这归因于潜空间中质量与分辨率间的典型权衡：高度压缩的潜空间虽能增强对退化的鲁棒性，却会降低保真度；而更灵活的潜空间虽能更好捕捉复杂面部细节，但针对高度退化人脸难以通过现有技术优化。为解决上述问题，我们在VQGAN架构中引入扩散先验，专注于学习未受损潜嵌入的分布。借助该先验知识，我们基于退化对应项迭代恢复纯净嵌入。此外，为确保逆向扩散轨迹不偏离原始身份特征，我们训练独立的身份恢复网络，并利用其输出约束逆向扩散过程。具体而言，通过可学习潜掩码，我们将人脸识别网络的梯度施加于与像素空间中精细身份细节相关的潜特征子集上，其余特征保持原状。潜空间中感知质量与保真度的解耦，使我们能同时实现最优的生成质量与重建精度。我们在多个真实与合成数据集上的广泛评估验证了本方法的优越性。