Many top-down architectures for instance segmentation achieve significant success when trained and tested on pre-defined closed-world taxonomy. However, when deployed in the open world, they exhibit notable bias towards seen classes and suffer from significant performance drop. In this work, we propose a novel approach for open world instance segmentation called bottom-Up and top-Down Open-world Segmentation (UDOS) that combines classical bottom-up segmentation algorithms within a top-down learning framework. UDOS first predicts parts of objects using a top-down network trained with weak supervision from bottom-up segmentations. The bottom-up segmentations are class-agnostic and do not overfit to specific taxonomies. The part-masks are then fed into affinity-based grouping and refinement modules to predict robust instance-level segmentations. UDOS enjoys both the speed and efficiency from the top-down architectures and the generalization ability to unseen categories from bottom-up supervision. We validate the strengths of UDOS on multiple cross-category as well as cross-dataset transfer tasks from 5 challenging datasets including MS-COCO, LVIS, ADE20k, UVO and OpenImages, achieving significant improvements over state-of-the-art across the board. Our code and models are available on our project page.

翻译：许多用于实例分割的自顶向下架构在预定义的封闭世界分类体系上进行训练和测试时取得了显著成功。然而，当部署在开放世界中时，这些架构对已见类别表现出明显偏差，并遭受严重的性能下降。本文提出了一种新颖的开放世界实例分割方法，称为自底向上和自顶向下的开放世界分割（UDOS），该方法将经典的自底向上分割算法整合到自顶向下学习框架中。UDOS首先使用自顶向下网络预测物体的部件，该网络通过自底向上分割的弱监督进行训练。自底向上分割与类别无关，不会过拟合特定分类体系。随后，部件掩码被输入基于亲和度的分组与精化模块，以预测鲁棒的实例级分割。UDOS既享有自顶向下架构的速度与效率，又具备自底向上监督对未见类别的泛化能力。我们在5个具有挑战性的数据集（包括MS-COCO、LVIS、ADE20k、UVO和OpenImages）上验证了UDOS在跨类别和跨数据集迁移任务中的优势，在所有任务上均取得了较现有最优方法的显著改进。我们的代码和模型可在项目页面获取。