We introduce uncertainty-aware object instance segmentation (UncOS) and demonstrate its usefulness for embodied interactive segmentation. To deal with uncertainty in robot perception, we propose a method for generating a hypothesis distribution of object segmentation. We obtain a set of region-factored segmentation hypotheses together with confidence estimates by making multiple queries of large pre-trained models. This process can produce segmentation results that achieve state-of-the-art performance on unseen object segmentation problems. The output can also serve as input to a belief-driven process for selecting robot actions to perturb the scene to reduce ambiguity. We demonstrate the effectiveness of this method in real-robot experiments. Website: https://sites.google.com/view/embodied-uncertain-seg

翻译：我们提出了不确定性感知物体实例分割方法（UncOS），并展示了其在具身交互式分割中的实用性。为处理机器人感知中的不确定性，我们提出了一种生成物体分割假设分布的方法。通过对大规模预训练模型进行多次查询，我们获得了一组区域分解的分割假设及其置信度估计。该过程能够生成在未见物体分割问题上达到最先进性能的分割结果。输出结果还可作为信念驱动过程的输入，用于选择机器人动作以扰动场景从而降低歧义。我们在真实机器人实验中验证了该方法的有效性。项目网站：https://sites.google.com/view/embodied-uncertain-seg