Volumetric video based on Neural Radiance Field (NeRF) holds vast potential for various 3D applications, but its substantial data volume poses significant challenges for compression and transmission. Current NeRF compression lacks the flexibility to adjust video quality and bitrate within a single model for various network and device capacities. To address these issues, we propose HPC, a novel hierarchical progressive volumetric video coding framework achieving variable bitrate using a single model. Specifically, HPC introduces a hierarchical representation with a multi-resolution residual radiance field to reduce temporal redundancy in long-duration sequences while simultaneously generating various levels of detail. Then, we propose an end-to-end progressive learning approach with a multi-rate-distortion loss function to jointly optimize both hierarchical representation and compression. Our HPC trained only once can realize multiple compression levels, while the current methods need to train multiple fixed-bitrate models for different rate-distortion (RD) tradeoffs. Extensive experiments demonstrate that HPC achieves flexible quality levels with variable bitrate by a single model and exhibits competitive RD performance, even outperforming fixed-bitrate models across various datasets.
翻译:基于神经辐射场(NeRF)的体视频在各类三维应用中具有巨大潜力,但其庞大的数据量对压缩与传输提出了重大挑战。现有的NeRF压缩方法缺乏灵活性,难以在单一模型中根据不同的网络与设备能力调整视频质量与码率。为解决这些问题,我们提出了HPC,一种新颖的分层渐进式体视频编码框架,能够使用单一模型实现可变码率。具体而言,HPC引入了一种分层表示结构,通过多分辨率残差辐射场来减少长时序列的时间冗余,同时生成多种细节层次。随后,我们提出一种端到端的渐进式学习方法,结合多码率-失真损失函数,对分层表示与压缩过程进行联合优化。我们的HPC仅需训练一次即可实现多个压缩等级,而现有方法需要为不同的码率-失真(RD)权衡训练多个固定码率模型。大量实验表明,HPC通过单一模型实现了可变码率下的灵活质量等级,并展现出具有竞争力的RD性能,甚至在多个数据集上优于固定码率模型。