Recent techniques for real-time view synthesis have rapidly advanced in fidelity and speed, and modern methods are capable of rendering near-photorealistic scenes at interactive frame rates. At the same time, a tension has arisen between explicit scene representations amenable to rasterization and neural fields built on ray marching, with state-of-the-art instances of the latter surpassing the former in quality while being prohibitively expensive for real-time applications. In this work, we introduce SMERF, a view synthesis approach that achieves state-of-the-art accuracy among real-time methods on large scenes with footprints up to 300 m$^2$ at a volumetric resolution of 3.5 mm$^3$. Our method is built upon two primary contributions: a hierarchical model partitioning scheme, which increases model capacity while constraining compute and memory consumption, and a distillation training strategy that simultaneously yields high fidelity and internal consistency. Our approach enables full six degrees of freedom (6DOF) navigation within a web browser and renders in real-time on commodity smartphones and laptops. Extensive experiments show that our method exceeds the current state-of-the-art in real-time novel view synthesis by 0.78 dB on standard benchmarks and 1.78 dB on large scenes, renders frames three orders of magnitude faster than state-of-the-art radiance field models, and achieves real-time performance across a wide variety of commodity devices, including smartphones. We encourage the reader to explore these models in person at our project website: https://smerf-3d.github.io.

翻译：近期实时视图合成技术在保真度和速度方面取得了快速进展，现代方法能够以交互帧率渲染接近照片级真实的场景。与此同时，适用于光栅化的显式场景表示与基于光线步进的神经辐射场之间出现了矛盾：后者的最新成果在质量上超越前者，但因其高昂的计算成本而难以应用于实时场景。本文提出SMERF方法，该方法在处理覆盖300平方米、体素分辨率达3.5立方毫米的大规模场景时，在实时方法中达到了最先进的精度。我们的方法基于两大核心贡献：一是层次化模型分割方案，通过约束计算与内存消耗来提升模型容量；二是蒸馏训练策略，同步实现高保真度与内部一致性。该方法支持在网页浏览器中实现完整的六自由度导航，并在主流智能手机和笔记本电脑上实现实时渲染。大量实验表明，我们的方法在标准基准测试和大规模场景下，分别以0.78 dB和1.78 dB的优势超越当前实时新视角合成技术；帧率比最先进的辐射场模型快三个数量级，并在包括智能手机在内的多种主流设备上实现实时性能。诚邀读者访问项目网站（https://smerf-3d.github.io）亲自探索这些模型。