We propose the task of Panoptic Scene Completion (PSC) which extends the recently popular Semantic Scene Completion (SSC) task with instance-level information to produce a richer understanding of the 3D scene. Our PSC proposal utilizes a hybrid mask-based technique on the non-empty voxels from sparse multi-scale completions. Whereas the SSC literature overlooks uncertainty which is critical for robotics applications, we instead propose an efficient ensembling to estimate both voxel-wise and instance-wise uncertainties along PSC. This is achieved by building on a multi-input multi-output (MIMO) strategy, while improving performance and yielding better uncertainty for little additional compute. Additionally, we introduce a technique to aggregate permutation-invariant mask predictions. Our experiments demonstrate that our method surpasses all baselines in both Panoptic Scene Completion and uncertainty estimation on three large-scale autonomous driving datasets. Our code and data are available at https://astra-vision.github.io/PaSCo .

翻译：摘要：我们提出全景场景补全（Panoptic Scene Completion, PSC）任务，该任务在近期流行的语义场景补全（Semantic Scene Completion, SSC）基础上引入实例级信息，以实现对三维场景更丰富的理解。我们的PSC方法利用基于混合掩膜的技术处理多尺度稀疏补全中的非空体素。针对SSC文献中忽视对机器人应用至关重要的不确定性问题，我们提出一种高效集成方法，沿PSC流程同时估计体素级和实例级的不确定性。这通过构建多输入多输出（MIMO）策略实现，在提升性能的同时以极低额外计算量获得更优的不确定性估计。此外，我们引入一种聚合置换不变掩膜预测的技术。实验表明，我们的方法在三大规模自动驾驶数据集的全景场景补全与不确定性估计任务中均超越所有基线。代码与数据详见https://astra-vision.github.io/PaSCo。