The vision-based perception for autonomous driving has undergone a transformation from the bird-eye-view (BEV) representations to the 3D semantic occupancy. Compared with the BEV planes, the 3D semantic occupancy further provides structural information along the vertical direction. This paper presents OccFormer, a dual-path transformer network to effectively process the 3D volume for semantic occupancy prediction. OccFormer achieves a long-range, dynamic, and efficient encoding of the camera-generated 3D voxel features. It is obtained by decomposing the heavy 3D processing into the local and global transformer pathways along the horizontal plane. For the occupancy decoder, we adapt the vanilla Mask2Former for 3D semantic occupancy by proposing preserve-pooling and class-guided sampling, which notably mitigate the sparsity and class imbalance. Experimental results demonstrate that OccFormer significantly outperforms existing methods for semantic scene completion on SemanticKITTI dataset and for LiDAR semantic segmentation on nuScenes dataset. Code is available at \url{https://github.com/zhangyp15/OccFormer}.

翻译：基于视觉的自动驾驶感知正经历从鸟瞰图表示到三维语义占据的转变。相较于鸟瞰平面，三维语义占据进一步提供了垂直方向的结构信息。本文提出OccFormer，一种用于有效处理三维体素以进行语义占据预测的双路Transformer网络。OccFormer实现了对相机生成的三维体素特征的远程、动态且高效的编码。其核心思路是将繁重的三维处理分解为沿水平平面进行的局部与全局Transformer路径。对于占据解码器，我们通过提出保留池化与类别引导采样，将原始Mask2Former适配至三维语义占据任务，显著缓解了稀疏性与类别不平衡问题。实验结果表明，OccFormer在SemanticKITTI数据集上的语义场景补全任务以及nuScenes数据集上的激光雷达语义分割任务中均显著优于现有方法。代码开源地址：\url{https://github.com/zhangyp15/OccFormer}。