Recently, generative models for 3D objects are gaining much popularity in VR and augmented reality applications. Training such models using standard 3D representations, like voxels or point clouds, is challenging and requires complex tools for proper color rendering. In order to overcome this limitation, Neural Radiance Fields (NeRFs) offer a state-of-the-art quality in synthesizing novel views of complex 3D scenes from a small subset of 2D images. In the paper, we propose a generative model called HyperNeRFGAN, which uses hypernetworks paradigm to produce 3D objects represented by NeRF. Our GAN architecture leverages a hypernetwork paradigm to transfer gaussian noise into weights of NeRF model. The model is further used to render 2D novel views, and a classical 2D discriminator is utilized for training the entire GAN-based structure. Our architecture produces 2D images, but we use 3D-aware NeRF representation, which forces the model to produce correct 3D objects. The advantage of the model over existing approaches is that it produces a dedicated NeRF representation for the object without sharing some global parameters of the rendering component. We show the superiority of our approach compared to reference baselines on three challenging datasets from various domains.

翻译：近期，面向三维物体的生成模型在虚拟现实与增强现实应用中日益流行。使用标准三维表示（如体素或点云）训练此类模型具有挑战性，且需要复杂工具实现色彩渲染。为克服这一局限，神经辐射场（NeRF）提供了一种从少量二维图像子集合成复杂三维场景新视角的前沿方法。本文提出名为HyperNeRFGAN的生成模型，利用超网络范式生成由NeRF表示的三维物体。我们的生成对抗网络架构采用超网络范式将高斯噪声映射至NeRF模型参数，进而通过该模型渲染二维新视角，并利用经典二维判别器训练整个基于GAN的结构。尽管架构输出二维图像，但通过采用三维感知的NeRF表示，促使模型生成正确的三维物体。相较于现有方法，本模型优势在于为物体生成专属NeRF表示，无需共享渲染组件的全局参数。我们在三个跨领域具有挑战性的数据集上验证了本方法相较于参考基线的优越性能。