Open World Object Detection(OWOD) addresses realistic scenarios where unseen object classes emerge, enabling detectors trained on known classes to detect unknown objects and incrementally incorporate the knowledge they provide. While existing OWOD methods primarily focus on detecting unknown objects, they often overlook the rich semantic relationships between detected objects, which are essential for scene understanding and applications in open-world environments (e.g., open-world tracking and novel class discovery). In this paper, we extend the OWOD framework to jointly detect unknown objects and learn semantically rich instance embeddings, enabling the detector to capture fine-grained semantic relationships between instances. To this end, we propose two modules that leverage the rich and generalizable knowledge of Vision Foundation Models(VFM). First, the Unknown Box Refine Module uses instance masks from the Segment Anything Model to accurately localize unknown objects. The Embedding Transfer Module then distills instance-wise semantic similarities from VFM features to the detector's embeddings via a relaxed contrastive loss, enabling the detector to learn a semantically meaningful and generalizable instance feature. Extensive experiments show that our method significantly improves both unknown object detection and instance embedding quality, while also enhancing performance in downstream tasks such as open-world tracking.

翻译：开放世界目标检测旨在应对现实场景中不断出现未知物体类别的挑战，使在已知类别上训练的检测器能够检测未知物体，并逐步吸收其提供的知识。现有的开放世界目标检测方法主要关注未知物体的检测，但往往忽略了检测到的物体之间丰富的语义关联，而这些关联对于开放世界环境下的场景理解与应用至关重要。本文中，我们扩展了开放世界目标检测框架，使其能够同时检测未知物体并学习语义丰富的实例嵌入，从而使检测器能够捕捉实例间细粒度的语义关联。为此，我们提出了两个模块，以利用视觉基础模型丰富且可泛化的知识。首先，未知框优化模块利用Segment Anything Model生成的实例掩码来精确定位未知物体。随后，嵌入迁移模块通过一种松弛的对比损失，将视觉基础模型特征中的实例级语义相似性知识蒸馏到检测器的嵌入中，使检测器能够学习到语义上有意义且可泛化的实例特征。大量实验表明，我们的方法显著提升了未知物体检测和实例嵌入的质量，同时也在开放世界跟踪等下游任务中增强了性能。