The efficient representation, transmission, and reconstruction of three-dimensional (3D) contents are becoming increasingly important for sixth-generation (6G) networks that aim to merge virtual and physical worlds for offering immersive communication experiences. Neural radiance field (NeRF) and 3D Gaussian splatting (3D-GS) have recently emerged as two promising 3D representation techniques based on radiance field rendering, which are able to provide photorealistic rendering results for complex scenes. Therefore, embracing NeRF and 3D-GS in 6G networks is envisioned to be a prominent solution to support emerging 3D applications with enhanced quality of experience. This paper provides a comprehensive overview on the integration of NeRF and 3D-GS in 6G. First, we review the basics of the radiance field rendering techniques, and highlight their applications and implementation challenges over wireless networks. Next, we consider the over-the-air training of NeRF and 3D-GS models over wireless networks by presenting various learning techniques. We particularly focus on the federated learning design over a hierarchical device-edge-cloud architecture, which is suitable for exploiting distributed data and computing resources over 6G networks to train large models representing large-scale scenes. Then, we consider the over-the-air rendering of NeRF and 3D-GS models at wireless network edge. We present three practical rendering architectures, namely local, remote, and co-rendering, respectively, and provide model compression approaches to facilitate the transmission of radiance field models for rendering. We also present rendering acceleration approaches and joint computation and communication designs to enhance the rendering efficiency. In a case study, we propose a new semantic communication enabled 3D content transmission design.
翻译:三维内容的高效表示、传输与重建对于旨在融合虚拟与物理世界以提供沉浸式通信体验的第六代网络正变得日益重要。神经辐射场与3D高斯泼溅作为基于辐射场渲染的两种新兴三维表示技术,能够为复杂场景提供逼真的渲染效果。因此,在6G网络中融合NeRF与3D-GS技术,有望成为支撑新兴三维应用并提升体验质量的突出解决方案。本文全面综述了NeRF与3D-GS在6G网络中的融合路径。首先,我们回顾辐射场渲染技术的基础原理,重点阐述其在无线网络中的应用场景与实施挑战。其次,我们探讨通过无线网络进行NeRF与3D-GS模型的空中训练,系统阐述多种学习技术。特别聚焦于分层设备-边缘-云架构下的联邦学习设计,该架构适用于利用6G网络的分布式数据与计算资源来训练表征大规模场景的巨量模型。随后,我们研究在无线网络边缘进行NeRF与3D-GS模型的空中渲染。我们提出三种实用渲染架构——本地渲染、远程渲染与协同渲染,并提供促进辐射场模型传输的模型压缩方法。同时阐述渲染加速技术及计算与通信联合设计方案以提升渲染效率。在案例研究中,我们提出一种基于语义通信的新型三维内容传输设计方案。