Human driver can easily describe the complex traffic scene by visual system. Such an ability of precise perception is essential for driver's planning. To achieve this, a geometry-aware representation that quantizes the physical 3D scene into structured grid map with semantic labels per cell, termed as 3D Occupancy, would be desirable. Compared to the form of bounding box, a key insight behind occupancy is that it could capture the fine-grained details of critical obstacles in the scene, and thereby facilitate subsequent tasks. Prior or concurrent literature mainly concentrate on a single scene completion task, where we might argue that the potential of this occupancy representation might obsess broader impact. In this paper, we propose OccNet, a multi-view vision-centric pipeline with a cascade and temporal voxel decoder to reconstruct 3D occupancy. At the core of OccNet is a general occupancy embedding to represent 3D physical world. Such a descriptor could be applied towards a wide span of driving tasks, including detection, segmentation and planning. To validate the effectiveness of this new representation and our proposed algorithm, we propose OpenOcc, the first dense high-quality 3D occupancy benchmark built on top of nuScenes. Empirical experiments show that there are evident performance gain across multiple tasks, e.g., motion planning could witness a collision rate reduction by 15%-58%, demonstrating the superiority of our method.

翻译：人类驾驶员能够通过视觉系统轻松描述复杂的交通场景。这种精确感知能力对于驾驶员的规划至关重要。为实现这一目标，一种将物理3D场景量化为结构化网格地图、每个网格单元带有语义标签的几何感知表示——称为3D占据——是理想的。与边界框形式相比，占据表示的关键洞察在于：它能捕捉场景中关键障碍物的细粒度细节，从而促进后续任务。现有或同期文献主要集中于单一场景补全任务，但我们认为这种占据表示的潜力可能具有更广泛的影响。本文提出OccNet，一种基于多视图的视觉中心流水线，采用级联时序体素解码器重建3D占据。OccNet的核心是一种通用占据嵌入，用于表示3D物理世界。这种描述符可应用于广泛的驾驶任务，包括检测、分割和规划。为验证这种新表示及所提算法的有效性，我们构建了OpenOcc——基于nuScenes的首个密集高质量3D占据基准。实验表明，多个任务均取得显著性能提升，例如运动规划的碰撞率降低15%-58%，证明了我们方法的优越性。