Real-world image manipulation has achieved fantastic progress in recent years. GAN inversion, which aims to map the real image to the latent code faithfully, is the first step in this pipeline. However, existing GAN inversion methods fail to achieve high reconstruction quality and fast inference at the same time. In addition, existing methods are built on 2D GANs and lack explicitly mechanisms to enforce multi-view consistency.In this work, we present a novel meta-auxiliary framework, while leveraging the newly developed 3D GANs as generator. The proposed method adopts a two-stage strategy. In the first stage, we invert the input image to an editable latent code using off-the-shelf inversion techniques. The auxiliary network is proposed to refine the generator parameters with the given image as input, which both predicts offsets for weights of convolutional layers and sampling positions of volume rendering. In the second stage, we perform meta-learning to fast adapt the auxiliary network to the input image, then the final reconstructed image is synthesized via the meta-learned auxiliary network. Extensive experiments show that our method achieves better performances on both inversion and editing tasks.

翻译：现实世界图像处理近年来取得了显著进展。生成对抗网络逆变换作为该流程的首要步骤，旨在将真实图像精确映射至潜在编码空间。然而，现有生成对抗网络逆变换方法无法同时实现高质量重建与快速推理。此外，现有方法多基于二维生成对抗网络构建，缺乏显式机制来保障多视角一致性。本文提出一种新颖的元辅助框架，并创新性地采用最新开发的三维生成对抗网络作为生成器。所提方法采用两阶段策略：第一阶段，利用现成逆变换技术将输入图像反演为可编辑潜在编码；设计辅助网络通过输入图像引导生成器参数优化，同步预测卷积层权重偏移量与体渲染采样位置。第二阶段，通过元学习实现辅助网络对输入图像的快速自适应，最终由元学习优化的辅助网络合成重建图像。大量实验证明，本方法在逆变换与编辑任务中均取得更优性能。