Occupancy prediction plays a pivotal role in the realm of autonomous driving. Previous methods typically constructs a dense 3D volume, neglecting the inherent sparsity of the scene, which results in a high computational cost. Furthermore, these methods are limited to semantic occupancy and fail to differentiate between distinct instances. To exploit the sparsity property and ensure instance-awareness, we introduce a novel fully sparse panoptic occupancy network, termed SparseOcc. SparseOcc initially reconstructs a sparse 3D representation from visual inputs. Subsequently, it employs sparse instance queries to predict each object instance from the sparse 3D representation. These instance queries interact with 2D features via mask-guided sparse sampling, thereby circumventing the need for costly dense features or global attention. Additionally, we have established the first-ever vision-centric panoptic occupancy benchmark. SparseOcc demonstrates its efficacy on the Occ3D-nus dataset by achieving a mean Intersection over Union (mIoU) of 26.0, while maintaining a real-time inference speed of 25.4 FPS. By incorporating temporal modeling from the preceding 8 frames, SparseOcc further improves its performance, achieving 30.9 mIoU without whistles and bells. Code will be made available.

翻译：占用预测在自动驾驶领域发挥着关键作用。现有方法通常构建稠密的三维体素，忽略了场景的固有稀疏性，导致计算成本高昂。此外，这些方法仅限于语义占用，无法区分不同实例。为利用稀疏性并实现实例感知，我们提出了一种全新的全稀疏全景占用网络SparseOcc。SparseOcc首先从视觉输入重建稀疏三维表示，随后通过稀疏实例查询从该稀疏三维表示中预测每个物体实例。这些实例查询通过遮挡引导的稀疏采样与二维特征交互，从而避免了对昂贵稠密特征或全局注意力的需求。同时，我们建立了首个基于视觉的全景占用基准。SparseOcc在Occ3D-nus数据集上展现出其有效性，实现了26.0的平均交并比（mIoU），同时保持25.4 FPS的实时推理速度。通过融入前8帧的时间建模，SparseOcc在不依赖任何技巧的情况下进一步提升了性能，达到30.9 mIoU。代码将开源。