Recent advances in neural radiance fields (NeRFs) achieve state-of-the-art novel view synthesis and facilitate dense estimation of scene properties. However, NeRFs often fail for large, unbounded scenes that are captured under very sparse views with the scene content concentrated far away from the camera, as is typical for field robotics applications. In particular, NeRF-style algorithms perform poorly: (1) when there are insufficient views with little pose diversity, (2) when scenes contain saturation and shadows, and (3) when finely sampling large unbounded scenes with fine structures becomes computationally intensive. This paper proposes CLONeR, which significantly improves upon NeRF by allowing it to model large outdoor driving scenes that are observed from sparse input sensor views. This is achieved by decoupling occupancy and color learning within the NeRF framework into separate Multi-Layer Perceptrons (MLPs) trained using LiDAR and camera data, respectively. In addition, this paper proposes a novel method to build differentiable 3D Occupancy Grid Maps (OGM) alongside the NeRF model, and leverage this occupancy grid for improved sampling of points along a ray for volumetric rendering in metric space. Through extensive quantitative and qualitative experiments on scenes from the KITTI dataset, this paper demonstrates that the proposed method outperforms state-of-the-art NeRF models on both novel view synthesis and dense depth prediction tasks when trained on sparse input data.

翻译：神经辐射场（NeRF）的最新进展实现了最先进的新视角合成，并促进了场景属性的密集估计。然而，NeRF在处理大规模无界场景时往往表现不佳，这类场景通常在极稀疏视角下拍摄，且场景内容集中在远离相机的位置，正如野外机器人应用中的典型情况。具体而言，NeRF类算法在以下情况下性能较差：（1）视角不足且姿态多样性低时；（2）场景存在饱和与阴影时；（3）对包含精细结构的大规模无界场景进行精细采样时计算开销巨大。本文提出CLONeR方法，通过允许NeRF建模从稀疏输入传感器视角观察的大型户外驾驶场景，显著提升了其性能。该方法将NeRF框架中的占用学习与颜色学习解耦为独立的多层感知机（MLP），分别利用激光雷达数据和相机数据进行训练。此外，本文提出一种新方法，在NeRF模型旁边构建可微的三维占用栅格地图（OGM），并利用该占用栅格改进沿射线采样点的策略，以在度量空间中进行体素渲染。通过在KITTI数据集场景上进行的广泛定量与定性实验，本文证明所提方法在稀疏输入数据训练条件下，在新视角合成与密集深度预测任务上均优于现有最先进的NeRF模型。