Neural radiance fields (NeRFs) enable high-quality novel view synthesis, but their prohibitively high computational complexity limits deployability, especially on resource-constrained platforms. To enable practical usage of NeRFs, quality tuning is essential to reduce computational complexity, akin to adjustable graphics settings in video games. However while existing solutions strive for efficiency, they use one-size-fits-all architectures regardless of scene complexity, although the same architecture may be unnecessarily large for simple scenes but insufficient for complex ones. Thus as NeRFs become more widely used for 3D visualization, there is a need to dynamically optimize the neural network component of NeRFs to achieve a balance between computational complexity and specific targets for synthesis quality. Addressing this gap, we introduce NAS-NeRF: a generative neural architecture search strategy uniquely tailored to generate NeRF architectures on a per-scene basis by optimizing the trade-off between complexity and performance, while adhering to constraints on computational budget and minimum synthesis quality. Our experiments on the Blender synthetic dataset show the proposed NAS-NeRF can generate architectures up to 5.74$\times$ smaller, with 4.19$\times$ fewer FLOPs, and 1.93$\times$ faster on a GPU than baseline NeRFs, without suffering a drop in SSIM. Furthermore, we illustrate that NAS-NeRF can also achieve architectures up to 23$\times$ smaller, 22$\times$ fewer FLOPs, and 4.7$\times$ faster than baseline NeRFs with only a 5.3\% average SSIM drop. The source code for our work is also made publicly available at https://saeejithnair.github.io/NAS-NeRF.
翻译:神经辐射场(NeRF)实现了高质量的新视角合成,但其极高的计算复杂度限制了其可部署性,尤其在资源受限平台上尤为突出。为促进NeRF的实际应用,质量调优对降低计算复杂度至关重要,类似于电子游戏中的可调图形设置。然而,现有解决方案虽追求效率,却采用"一刀切"架构而不考虑场景复杂度——相同架构可能对简单场景过于冗余,对复杂场景又显不足。随着NeRF在三维可视化领域的广泛应用,亟需动态优化其神经网络组件,以在计算复杂度与合成质量的特定目标间取得平衡。针对这一空白,我们提出NAS-NeRF:一种专为按场景生成NeRF架构而设计的生成式神经架构搜索策略,通过优化复杂度与性能的折中,同时满足计算预算与最低合成质量的约束。在Blender合成数据集上的实验表明,与基线NeRF相比,NAS-NeRF生成的架构可缩小5.74倍、FLOPs减少4.19倍、GPU加速1.93倍,且SSIM指标未下降。此外,NAS-NeRF还能实现架构缩小23倍、FLOPs减少22倍、GPU加速4.7倍,而SSIM平均仅下降5.3%。本工作源代码已在https://saeejithnair.github.io/NAS-NeRF 公开。