Object detection has been expanded from a limited number of categories to open vocabulary. Moving forward, a complete intelligent vision system requires understanding more fine-grained object descriptions, object parts. In this paper, we propose a detector with the ability to predict both open-vocabulary objects and their part segmentation. This ability comes from two designs. First, we train the detector on the joint of part-level, object-level and image-level data to build the multi-granularity alignment between language and image. Second, we parse the novel object into its parts by its dense semantic correspondence with the base object. These two designs enable the detector to largely benefit from various data sources and foundation models. In open-vocabulary part segmentation experiments, our method outperforms the baseline by 3.3$\sim$7.3 mAP in cross-dataset generalization on PartImageNet, and improves the baseline by 7.3 novel AP$_{50}$ in cross-category generalization on Pascal Part. Finally, we train a detector that generalizes to a wide range of part segmentation datasets while achieving better performance than dataset-specific training.

翻译：目标检测已从有限类别扩展至开放词汇。更进一步，完整的智能视觉系统需要理解更细粒度的物体描述——物体部件。本文提出一种能够同时预测开放词汇物体及其部件分割的检测器。这一能力源于两个设计：首先，我们在部件级、物体级和图像级数据的联合上训练检测器，以构建语言与图像之间的多粒度对齐；其次，我们通过新物体与基物体之间的密集语义对应关系，将新物体解析为部件。这两个设计使检测器能够充分利用多种数据源和基础模型。在开放词汇部件分割实验中，我们的方法在PartImageNet数据集上的跨数据集泛化性能比基线高出3.3~7.3 mAP，并在Pascal Part数据集上的跨类别泛化性能将基线novel AP50提升7.3。最终，我们训练的检测器能泛化到广泛的部件分割数据集，同时性能优于针对特定数据集的训练。