Modeling 3D scenes by volumetric feature grids is one of the promising directions of neural approximations to improve Neural Radiance Fields (NeRF). Instant-NGP (INGP) introduced multi-resolution hash encoding from a lookup table of trainable feature grids which enabled learning high-quality neural graphics primitives in a matter of seconds. However, this improvement came at the cost of higher storage size. In this paper, we address this challenge by introducing instant learning of compression-aware NeRF features (CAwa-NeRF), that allows exporting the zip compressed feature grids at the end of the model training with a negligible extra time overhead without changing neither the storage architecture nor the parameters used in the original INGP paper. Nonetheless, the proposed method is not limited to INGP but could also be adapted to any model. By means of extensive simulations, our proposed instant learning pipeline can achieve impressive results on different kinds of static scenes such as single object masked background scenes and real-life scenes captured in our studio. In particular, for single object masked background scenes CAwa-NeRF compresses the feature grids down to 6% (1.2 MB) of the original size without any loss in the PSNR (33 dB) or down to 2.4% (0.53 MB) with a slight virtual loss (32.31 dB).
翻译:通过体素特征网格建模三维场景是改进神经辐射场(NeRF)的神经近似方法中的重要方向之一。Instant-NGP(INGP)通过可训练特征网格查找表引入多分辨率哈希编码,使得能够在数秒内学习高质量神经图形基元。然而,这一改进以更高的存储开销为代价。本文通过引入压缩感知NeRF特征的即时学习(CAwa-NeRF)来解决这一挑战,该方法允许在模型训练结束时导出zip压缩后的特征网格,且仅增加极小的额外时间开销,无需改变原始INGP论文中的存储架构或参数。但所提方法并不局限于INGP,也可适配于任何模型。通过大量仿真实验,我们提出的即时学习流程在不同类型的静态场景(如单物体掩膜背景场景和工作室采集的真实场景)上均能取得显著效果。具体而言,对于单物体掩膜背景场景,CAwa-NeRF可将特征网格压缩至原始尺寸的6%(1.2 MB),且PSNR(33 dB)完全无损;或压缩至2.4%(0.53 MB),仅产生微小虚拟损失(32.31 dB)。