Recent advances in Neural Radiance Fields (NeRFs) treat the problem of novel view synthesis as Sparse Radiance Field (SRF) optimization using sparse voxels for efficient and fast rendering (plenoxels,InstantNGP). In order to leverage machine learning and adoption of SRFs as a 3D representation, we present SPARF, a large-scale ShapeNet-based synthetic dataset for novel view synthesis consisting of $\sim$ 17 million images rendered from nearly 40,000 shapes at high resolution (400 X 400 pixels). The dataset is orders of magnitude larger than existing synthetic datasets for novel view synthesis and includes more than one million 3D-optimized radiance fields with multiple voxel resolutions. Furthermore, we propose a novel pipeline (SuRFNet) that learns to generate sparse voxel radiance fields from only few views. This is done by using the densely collected SPARF dataset and 3D sparse convolutions. SuRFNet employs partial SRFs from few/one images and a specialized SRF loss to learn to generate high-quality sparse voxel radiance fields that can be rendered from novel views. Our approach achieves state-of-the-art results in the task of unconstrained novel view synthesis based on few views on ShapeNet as compared to recent baselines. The SPARF dataset will be made public with the code and models on the project website https://abdullahamdi.com/sparf/ .

翻译：摘要：神经辐射场（NeRF）的最新进展将新视角合成问题转化为稀疏辐射场（SRF）优化，利用稀疏体素实现高效快速渲染（如plenoxels、InstantNGP）。为促进机器学习方法的应用并采用SRF作为三维表征，我们提出了SPARF——一个基于ShapeNet的大规模合成数据集，专门用于新视角合成。该数据集包含从近4万个形状渲染出的约1700万张高分辨率（400×400像素）图像，规模比现有用于新视角合成的合成数据集高出数个数量级，并提供超过一百万个具有多体素分辨率的优化三维辐射场。此外，我们提出了一种新型流水线（SuRFNet），该模型仅需少量视角即可学习生成稀疏体素辐射场，这一目标通过利用密集采集的SPARF数据集和三维稀疏卷积实现。SuRFNet采用来自极少量（单张或多张）图像的部分SRF以及特定的SRF损失函数，学习生成高质量且可从新视角渲染的稀疏体素辐射场。与近期基线方法相比，本方法在基于少量视图的无约束新视角合成任务中（基于ShapeNet数据）达到了最优结果。SPARF数据集将与代码和模型一同在项目网站https://abdullahamdi.com/sparf/ 上公开发布。