Generative Neural Radiance Fields (NeRFs) have demonstrated remarkable proficiency in synthesizing multi-view images by learning the distribution of a set of unposed images. Despite the aptitude of existing generative NeRFs in generating 3D-consistent high-quality random samples within data distribution, the creation of a 3D representation of a singular input image remains a formidable challenge. In this manuscript, we introduce ZIGNeRF, an innovative model that executes zero-shot Generative Adversarial Network (GAN) inversion for the generation of multi-view images from a single out-of-domain image. The model is underpinned by a novel inverter that maps out-of-domain images into the latent code of the generator manifold. Notably, ZIGNeRF is capable of disentangling the object from the background and executing 3D operations such as 360-degree rotation or depth and horizontal translation. The efficacy of our model is validated using multiple real-image datasets: Cats, AFHQ, CelebA, CelebA-HQ, and CompCars.
翻译:生成神经辐射场(NeRFs)通过学习一组无位姿图像的数据分布,在合成多视角图像方面展现了卓越的能力。尽管现有生成式NeRF在数据分布内生成三维一致的高质量随机样本方面表现出色,但针对单张输入图像创建三维表示仍是一项艰巨挑战。本文提出ZIGNeRF这一创新模型,通过执行零样本生成对抗网络(GAN)反演,实现从单张域外图像生成多视角图像。该模型的核心是一种新型反演器,可将域外图像映射至生成器流形的潜在编码中。值得注意的是,ZIGNeRF能够将物体与背景解耦,并执行360度旋转、深度平移及水平平移等三维操作。我们通过多个真实图像数据集(Cats、AFHQ、CelebA、CelebA-HQ及CompCars)验证了模型的有效性。