Advances in 3D-aware generative models have pushed the boundary of image synthesis with explicit camera control. To achieve high-resolution image synthesis, several attempts have been made to design efficient generators, such as hybrid architectures with both 3D and 2D components. However, such a design compromises multiview consistency, and the design of a pure 3D generator with high resolution is still an open problem. In this work, we present Generative Volumetric Primitives (GVP), the first pure 3D generative model that can sample and render 512-resolution images in real-time. GVP jointly models a number of volumetric primitives and their spatial information, both of which can be efficiently generated via a 2D convolutional network. The mixture of these primitives naturally captures the sparsity and correspondence in the 3D volume. The training of such a generator with a high degree of freedom is made possible through a knowledge distillation technique. Experiments on several datasets demonstrate superior efficiency and 3D consistency of GVP over the state-of-the-art.

翻译：三维感知生成模型的进展推动了具备显式相机控制的图像合成边界。为实现高分辨率图像合成，研究者尝试设计高效生成器，例如包含三维和二维组件的混合架构。然而，此类设计会损害多视图一致性，而纯三维高分辨率生成器的设计仍是一个开放问题。本文提出生成式体积基元（GVP），这是首个能够实时采样并渲染512分辨率图像的纯三维生成模型。GVP联合建模多个体积基元及其空间信息，两者均可通过二维卷积网络高效生成。这些基元的混合自然捕捉了三维体积中的稀疏性与对应关系。通过知识蒸馏技术，我们实现了具有高度自由度的生成器训练。在多个数据集上的实验表明，GVP在效率和三维一致性上均优于现有最先进方法。