The efficient representation, transmission, and reconstruction of three-dimensional (3D) contents are becoming increasingly important for sixth-generation (6G) networks that aim to merge virtual and physical worlds for offering immersive communication experiences. Neural radiance field (NeRF) and 3D Gaussian splatting (3D-GS) have recently emerged as two promising 3D representation techniques based on radiance field rendering, which are able to provide photorealistic rendering results for complex scenes. Therefore, embracing NeRF and 3D-GS in 6G networks is envisioned to be a prominent solution to support emerging 3D applications with enhanced quality of experience. This paper provides a comprehensive overview on the integration of NeRF and 3D-GS in 6G. First, we review the basics of the radiance field rendering techniques, and highlight their applications and implementation challenges over wireless networks. Next, we consider the over-the-air training of NeRF and 3D-GS models over wireless networks by presenting various learning techniques. We particularly focus on the federated learning design over a hierarchical device-edge-cloud architecture. Then, we discuss three practical rendering architectures of NeRF and 3D-GS models at wireless network edge. We provide model compression approaches to facilitate the transmission of radiance field models, and present rendering acceleration approaches and joint computation and communication designs to enhance the rendering efficiency. In particular, we propose a new semantic communication enabled 3D content transmission design, in which the radiance field models are exploited as the semantic knowledge base to reduce the communication overhead for distributed inference. Furthermore, we present the utilization of radiance field rendering in wireless applications like radio mapping and radio imaging.
翻译:三维内容的高效表示、传输与重建对旨在融合虚拟与现实世界以提供沉浸式通信体验的第六代(6G)网络日益重要。神经辐射场(NeRF)和三维高斯泼溅(3D-GS)近期成为两种基于辐射场渲染的具有前景的三维表示技术,能够为复杂场景提供照片级渲染效果。因此,将NeRF和3D-GS融入6G网络被视为支持新兴三维应用、提升体验质量的突出解决方案。本文全面综述了NeRF与3D-GS在6G中的集成。首先,我们回顾了辐射场渲染技术的基本原理,并重点阐述其在无线网络中的应用及实现挑战。其次,我们通过介绍多种学习技术,探讨了在无线网络上对NeRF和3D-GS模型进行空中训练的方法,特别关注基于分层设备-边缘-云架构的联邦学习设计。随后,讨论了无线网络边缘部署NeRF和3D-GS模型的三种实用渲染架构。我们提出模型压缩方法以促进辐射场模型的传输,并给出渲染加速方法及联合计算与通信设计以提升渲染效率。特别地,我们提出一种新型语义通信驱动的三维内容传输设计,将辐射场模型作为语义知识库,以减少分布式推理的通信开销。此外,我们还展示了辐射场渲染在无线映射与无线成像等无线应用中的利用。