Recent techniques for real-time view synthesis have rapidly advanced in fidelity and speed, and modern methods are capable of rendering near-photorealistic scenes at interactive frame rates. At the same time, a tension has arisen between explicit scene representations amenable to rasterization and neural fields built on ray marching, with state-of-the-art instances of the latter surpassing the former in quality while being prohibitively expensive for real-time applications. In this work, we introduce SMERF, a view synthesis approach that achieves state-of-the-art accuracy among real-time methods on large scenes with footprints up to 300 m$^2$ at a volumetric resolution of 3.5 mm$^3$. Our method is built upon two primary contributions: a hierarchical model partitioning scheme, which increases model capacity while constraining compute and memory consumption, and a distillation training strategy that simultaneously yields high fidelity and internal consistency. Our approach enables full six degrees of freedom (6DOF) navigation within a web browser and renders in real-time on commodity smartphones and laptops. Extensive experiments show that our method exceeds the current state-of-the-art in real-time novel view synthesis by 0.78 dB on standard benchmarks and 1.78 dB on large scenes, renders frames three orders of magnitude faster than state-of-the-art radiance field models, and achieves real-time performance across a wide variety of commodity devices, including smartphones. We encourage readers to explore these models interactively at our project website: https://smerf-3d.github.io.

翻译：近年来，实时视图合成技术在保真度和速度方面发展迅速，现代方法已能以交互式帧率渲染近乎照片级真实的场景。与此同时，适用于光栅化的显式场景表示与基于光线步进的神经场之间出现了张力，后者的先进实例在质量上超越了前者，但其计算成本对于实时应用而言过于高昂。本研究提出了SMERF，这是一种视图合成方法，在占地面积达300平方米、体素分辨率为3.5立方毫米的大规模场景上，其精度达到了实时方法中的最先进水平。我们的方法基于两项核心贡献：一种分层模型分区方案，在约束计算和内存消耗的同时提升模型容量；以及一种蒸馏训练策略，能同时实现高保真度与内部一致性。我们的方法支持在网页浏览器中进行完整的六自由度（6DOF）导航，并能在商用智能手机和笔记本电脑上实时渲染。大量实验表明，我们的方法在标准基准测试中比当前最先进的实时新视图合成方法高出0.78 dB，在大场景中高出1.78 dB，渲染帧速比最先进的辐射场模型快三个数量级，并在包括智能手机在内的多种商用设备上实现了实时性能。我们鼓励读者通过我们的项目网站（https://smerf-3d.github.io）交互式探索这些模型。