Recent techniques for real-time view synthesis have rapidly advanced in fidelity and speed, and modern methods are capable of rendering near-photorealistic scenes at interactive frame rates. At the same time, a tension has arisen between explicit scene representations amenable to rasterization and neural fields built on ray marching, with state-of-the-art instances of the latter surpassing the former in quality while being prohibitively expensive for real-time applications. In this work, we introduce SMERF, a view synthesis approach that achieves state-of-the-art accuracy among real-time methods on large scenes with footprints up to 300 m$^2$ at a volumetric resolution of 3.5 mm$^3$. Our method is built upon two primary contributions: a hierarchical model partitioning scheme, which increases model capacity while constraining compute and memory consumption, and a distillation training strategy that simultaneously yields high fidelity and internal consistency. Our approach enables full six degrees of freedom (6DOF) navigation within a web browser and renders in real-time on commodity smartphones and laptops. Extensive experiments show that our method exceeds the current state-of-the-art in real-time novel view synthesis by 0.78 dB on standard benchmarks and 1.78 dB on large scenes, renders frames three orders of magnitude faster than state-of-the-art radiance field models, and achieves real-time performance across a wide variety of commodity devices, including smartphones. We encourage readers to explore these models interactively at our project website: https://smerf-3d.github.io.

翻译：摘要：近期实时视角合成技术在保真度和速度上取得了快速进展，现代方法已能在交互帧率下渲染接近照片级真实的场景。然而，适用于光栅化的显式场景表示与基于光线步进的神经场之间产生了矛盾——后者在质量上超越前者，但因计算成本过高而难以用于实时应用。本文提出SMERF，一种视角合成方法，在覆盖面积达300平方米、体素分辨率达3.5立方毫米的大规模场景中，实现了实时方法中领先的精度。该方法基于两大核心贡献：一是分层模型划分方案，在约束计算与内存消耗的同时提升模型容量；二是蒸馏训练策略，可同时保证高保真度与内部一致性。我们的方法支持在网页浏览器中实现完整六自由度（6DOF）导航，并在商用智能手机与笔记本电脑上实时渲染。大量实验表明，该方法在标准基准测试和大规模场景中分别以0.78分贝和1.78分贝的PSNR提升超越当前实时新视角合成技术，渲染速度比最先进的辐射场模型快三个数量级，且能在包括智能手机在内的多种商用设备上实现实时性能。欢迎读者通过项目网站（https://smerf-3d.github.io）互动探索这些模型。