Semantic occupancy has recently gained significant traction as a prominent 3D scene representation. However, most existing methods rely on large and costly datasets with fine-grained 3D voxel labels for training, which limits their practicality and scalability, increasing the need for self-monitored learning in this domain. In this work, we present a novel approach to occupancy estimation inspired by neural radiance field (NeRF) using only 2D labels, which are considerably easier to acquire. In particular, we employ differentiable volumetric rendering to predict depth and semantic maps and train a 3D network based on 2D supervision only. To enhance geometric accuracy and increase the supervisory signal, we introduce temporal rendering of adjacent time steps. Additionally, we introduce occupancy flow as a mechanism to handle dynamic objects in the scene and ensure their temporal consistency. Through extensive experimentation we demonstrate that 2D supervision only is sufficient to achieve state-of-the-art performance compared to methods using 3D labels, while outperforming concurrent 2D approaches. When combining 2D supervision with 3D labels, temporal rendering and occupancy flow we outperform all previous occupancy estimation models significantly. We conclude that the proposed rendering supervision and occupancy flow advances occupancy estimation and further bridges the gap towards self-supervised learning in this domain.

翻译：语义占据作为突出的三维场景表示方法近期备受关注。然而现有方法大多依赖带有细粒度三维体素标注的大规模高成本数据集进行训练，这限制了其实用性与可扩展性，亟需在该领域引入自监督学习。本文提出一种受神经辐射场启发的全新占据估计方法，仅利用更易获取的二维标签，具体通过可微体素渲染预测深度图与语义图，在仅使用二维监督信号条件下训练三维网络。为增强几何精度并扩展监督信号，我们引入相邻时间步的时序渲染机制。同时提出占据流机制处理场景中动态目标并确保其时序一致性。大量实验表明，仅凭二维监督即可达到与使用三维标签方法相媲美的先进性能，同时优于现有二维方法。当结合二维监督与三维标签、时序渲染及占据流时，本方法显著超越所有先前的占据估计模型。我们得出结论：所提出的渲染监督与占据流机制推动了占据估计技术发展，进一步缩小了该领域与自监督学习之间的差距。