Online reconstructing and rendering of large-scale indoor scenes is a long-standing challenge. SLAM-based methods can reconstruct 3D scene geometry progressively in real time but can not render photorealistic results. While NeRF-based methods produce promising novel view synthesis results, their long offline optimization time and lack of geometric constraints pose challenges to efficiently handling online input. Inspired by the complementary advantages of classical 3D reconstruction and NeRF, we thus investigate marrying explicit geometric representation with NeRF rendering to achieve efficient online reconstruction and high-quality rendering. We introduce SurfelNeRF, a variant of neural radiance field which employs a flexible and scalable neural surfel representation to store geometric attributes and extracted appearance features from input images. We further extend the conventional surfel-based fusion scheme to progressively integrate incoming input frames into the reconstructed global neural scene representation. In addition, we propose a highly-efficient differentiable rasterization scheme for rendering neural surfel radiance fields, which helps SurfelNeRF achieve $10\times$ speedups in training and inference time, respectively. Experimental results show that our method achieves the state-of-the-art 23.82 PSNR and 29.58 PSNR on ScanNet in feedforward inference and per-scene optimization settings, respectively.

翻译：大规模室内场景的在线重建与渲染是一个长期存在的挑战。基于SLAM的方法虽能实时逐步重建三维场景几何结构，却无法生成逼真渲染结果。而基于NeRF的方法虽能产生令人满意的新视角合成效果，但其漫长的离线优化时间和缺乏几何约束限制了高效处理在线输入的能力。受经典三维重建与NeRF互补优势的启发，我们探索将显式几何表征与NeRF渲染相结合，以实现高效的在线重建与高质量渲染。为此提出SurfelNeRF——一种神经辐射场变体，其采用灵活且可扩展的神经面素表征来存储几何属性及从输入图像中提取的外观特征。我们进一步扩展传统面素融合方案，将输入帧逐步集成到重建的全局神经场景表征中。此外，我们提出一种用于神经面素辐射场渲染的高效可微分光栅化方案，使SurfelNeRF在训练和推理时间上分别实现10倍加速。实验结果表明，在ScanNet数据集上前馈推理和逐场景优化设置下，我们的方法分别达到了23.82 PSNR和29.58 PSNR的先进水平。