This paper proposes a novel approach for rendering a pre-trained Neural Radiance Field (NeRF) in real-time on resource-constrained devices. We introduce Re-ReND, a method enabling Real-time Rendering of NeRFs across Devices. Re-ReND is designed to achieve real-time performance by converting the NeRF into a representation that can be efficiently processed by standard graphics pipelines. The proposed method distills the NeRF by extracting the learned density into a mesh, while the learned color information is factorized into a set of matrices that represent the scene's light field. Factorization implies the field is queried via inexpensive MLP-free matrix multiplications, while using a light field allows rendering a pixel by querying the field a single time-as opposed to hundreds of queries when employing a radiance field. Since the proposed representation can be implemented using a fragment shader, it can be directly integrated with standard rasterization frameworks. Our flexible implementation can render a NeRF in real-time with low memory requirements and on a wide range of resource-constrained devices, including mobiles and AR/VR headsets. Notably, we find that Re-ReND can achieve over a 2.6-fold increase in rendering speed versus the state-of-the-art without perceptible losses in quality.

翻译：本文提出一种新颖方法，用于在资源受限设备上实时渲染预训练的神经辐射场（NeRF）。我们引入Re-ReND，一种实现跨设备实时渲染神经辐射场的方法。Re-ReND通过将NeRF转换为可被标准图形管线高效处理的表示形式，从而实现实时性能。该方法通过提取学习到的密度信息生成网格来蒸馏NeRF，同时将学习到的颜色信息分解为一组表示场景光场的矩阵。分解意味着可通过免MLP的廉价矩阵乘法查询该场，而使用光场则允许通过单次查询渲染像素——而使用辐射场时需数百次查询。由于该表示可通过片段着色器实现，因此可直接集成到标准光栅化框架中。我们灵活的实现在低内存需求下能在包括移动设备及AR/VR头戴设备在内的各类资源受限设备上实时渲染NeRF。值得注意的是，与现有最先进方法相比，Re-ReND在无感知质量损失情况下可实现超过2.6倍的渲染速度提升。