Generative Adversarial Networks (GANs) have emerged as powerful tools not only for high-quality image generation but also for real image editing through manipulation of their interpretable latent spaces. Recent advancements in GANs include the development of 3D-aware models such as EG3D, characterized by efficient triplane-based architectures enabling the reconstruction of 3D geometry from single images. However, scant attention has been devoted to providing an integrated framework for high-quality reference-based 3D-aware image editing within this domain. This study addresses this gap by exploring and demonstrating the effectiveness of EG3D's triplane space for achieving advanced reference-based edits, presenting a unique perspective on 3D-aware image editing through our novel pipeline. Our approach integrates the encoding of triplane features, spatial disentanglement and automatic localization of features in the triplane domain, and fusion learning for desired image editing. Moreover, our framework demonstrates versatility across domains, extending its effectiveness to animal face edits and partial stylization of cartoon portraits. The method shows significant improvements over relevant 3D-aware latent editing and 2D reference-based editing methods, both qualitatively and quantitatively. Project page: https://three-bee.github.io/triplane_edit
翻译:生成对抗网络(GANs)已成为不仅用于高质量图像生成,还可通过操纵其可解释的潜空间实现真实图像编辑的强大工具。GANs的最新进展包括开发出如EG3D等三维感知模型,其特色在于基于高效的三平面架构,能够从单张图像重建三维几何结构。然而,在该领域内,鲜有关注提供集成框架以实现高质量基于参考的三维感知图像编辑。本研究通过探索并证明EG3D的三平面空间在实现高级基于参考编辑中的有效性,填补了这一空白,并通过我们新颖的流水线提出了三维感知图像编辑的独特视角。我们的方法整合了三平面特征编码、空间解耦与三平面域中特征的自动定位,以及用于所需图像编辑的融合学习。此外,我们的框架展示了跨领域的多功能性,其有效性扩展至动物面部编辑和卡通肖像的部分风格化。该方法在定性和定量两方面均较相关三维感知潜编辑和二维基于参考的编辑方法有显著提升。项目页面:https://three-bee.github.io/triplane_edit