3D occupancy prediction is an important task for the robustness of vision-centric autonomous driving, which aims to predict whether each point is occupied in the surrounding 3D space. Existing methods usually require 3D occupancy labels to produce meaningful results. However, it is very laborious to annotate the occupancy status of each voxel. In this paper, we propose SelfOcc to explore a self-supervised way to learn 3D occupancy using only video sequences. We first transform the images into the 3D space (e.g., bird's eye view) to obtain 3D representation of the scene. We directly impose constraints on the 3D representations by treating them as signed distance fields. We can then render 2D images of previous and future frames as self-supervision signals to learn the 3D representations. We propose an MVS-embedded strategy to directly optimize the SDF-induced weights with multiple depth proposals. Our SelfOcc outperforms the previous best method SceneRF by 58.7% using a single frame as input on SemanticKITTI and is the first self-supervised work that produces reasonable 3D occupancy for surround cameras on nuScenes. SelfOcc produces high-quality depth and achieves state-of-the-art results on novel depth synthesis, monocular depth estimation, and surround-view depth estimation on the SemanticKITTI, KITTI-2015, and nuScenes, respectively. Code: https://github.com/huang-yh/SelfOcc.

翻译：三维占据预测是视觉中心自动驾驶鲁棒性的重要任务，旨在预测周围三维空间中每个点是否被占据。现有方法通常需要三维占据标签才能产生有意义的结果。然而，标记每个体素的占据状态非常耗费人力。本文提出SelfOcc，探索一种利用视频序列自监督学习三维占据的方法。我们首先将图像变换到三维空间（例如鸟瞰视角）以获取场景的三维表示。通过将三维表示视为符号距离场，我们直接对其施加约束。进而，我们可以渲染前后帧的二维图像作为自监督信号，用于学习三维表示。我们提出一种多视图立体嵌入策略，通过多个深度提议直接优化SDF诱导的权重。在SemanticKITTI数据集上，我们的SelfOcc以单帧输入相比先前最佳方法SceneRF提升58.7%，并且是首个在nuScenes数据集上为环视相机生成合理三维占据的自监督工作。SelfOcc能生成高质量深度，在SemanticKITTI、KITTI-2015和nuScenes数据集上分别在新颖深度合成、单目深度估计和环视深度估计任务中达到最先进结果。代码：https://github.com/huang-yh/SelfOcc。