Recently, a surge of high-quality 3D-aware GANs have been proposed, which leverage the generative power of neural rendering. It is natural to associate 3D GANs with GAN inversion methods to project a real image into the generator's latent space, allowing free-view consistent synthesis and editing, referred as 3D GAN inversion. Although with the facial prior preserved in pre-trained 3D GANs, reconstructing a 3D portrait with only one monocular image is still an ill-pose problem. The straightforward application of 2D GAN inversion methods focuses on texture similarity only while ignoring the correctness of 3D geometry shapes. It may raise geometry collapse effects, especially when reconstructing a side face under an extreme pose. Besides, the synthetic results in novel views are prone to be blurry. In this work, we propose a novel method to promote 3D GAN inversion by introducing facial symmetry prior. We design a pipeline and constraints to make full use of the pseudo auxiliary view obtained via image flipping, which helps obtain a robust and reasonable geometry shape during the inversion process. To enhance texture fidelity in unobserved viewpoints, pseudo labels from depth-guided 3D warping can provide extra supervision. We design constraints aimed at filtering out conflict areas for optimization in asymmetric situations. Comprehensive quantitative and qualitative evaluations on image reconstruction and editing demonstrate the superiority of our method.
翻译:近期,一系列高质量的三维感知生成对抗网络(3D-aware GANs)被提出,它们利用神经渲染的生成能力。将三维GAN与GAN逆映射方法关联,以将真实图像投影到生成器的潜在空间,从而实现自由视角一致合成与编辑(称为三维GAN逆映射)是自然之举。尽管预训练的三维GAN中保留了面部先验,但仅凭单目图像重建三维肖像仍是一个病态问题。直接应用二维GAN逆映射方法仅关注纹理相似性,而忽略三维几何形状的正确性,这可能导致几何坍塌效应,尤其是在极端姿态下重建侧脸时。此外,新视角下的合成结果容易模糊。本文提出一种新方法,通过引入面部对称先验来促进三维GAN逆映射。我们设计了一套流程与约束,充分利用通过图像翻转获得的伪辅助视图,从而在逆映射过程中获得稳健合理的几何形状。为增强未观测视角的纹理保真度,来自深度引导的三维扭曲的伪标签可提供额外监督。我们设计了针对非对称情况下冲突区域过滤优化的约束。在图像重建与编辑上的全面定量与定性评估表明了我们方法的优越性。