NeRF has significantly advanced 3D scene reconstruction, capturing intricate details across various environments. Existing methods have successfully leveraged radiance field baking to facilitate real-time rendering of small scenes. However, when applied to large-scale scenes, these techniques encounter significant challenges, struggling to provide a seamless real-time experience due to limited resources in computation, memory, and bandwidth. In this paper, we propose City-on-Web, which represents the whole scene by partitioning it into manageable blocks, each with its own Level-of-Detail, ensuring high fidelity, efficient memory management and fast rendering. Meanwhile, we carefully design the training and inference process such that the final rendering result on web is consistent with training. Thanks to our novel representation and carefully designed training/inference process, we are the first to achieve real-time rendering of large-scale scenes in resource-constrained environments. Extensive experimental results demonstrate that our method facilitates real-time rendering of large-scale scenes on a web platform, achieving 32FPS at 1080P resolution with an RTX 3060 GPU, while simultaneously achieving a quality that closely rivals that of state-of-the-art methods. Project page: https://ustc3dv.github.io/City-on-Web/
翻译:NeRF通过捕捉各种环境中的复杂细节,显著推进了3D场景重建。现有方法已成功利用辐射场烘焙实现小场景的实时渲染。然而,当应用于大规模场景时,这些技术面临重大挑战:在计算、内存和带宽资源受限的情况下,难以提供无缝的实时体验。本文提出城市网络(City-on-Web),该方法通过将整个场景划分为可管理的块(每个块具有独立细节层级)来实现场景表示,确保高保真度、高效内存管理与快速渲染。同时,我们精心设计了训练与推理流程,使网页端的最终渲染结果与训练保持一致性。得益于新颖的表示方式及精心设计的训练/推理流程,我们首次在资源受限环境中实现了大规模场景的实时渲染。大量实验表明,我们的方法可在网页平台实现大规模场景的实时渲染:在RTX 3060 GPU上达到1080P分辨率下32FPS的帧率,同时渲染质量与最先进方法水平相当。项目主页:https://ustc3dv.github.io/City-on-Web/