In this paper, we formally address universal object detection, which aims to detect every scene and predict every category. The dependence on human annotations, the limited visual information, and the novel categories in the open world severely restrict the universality of traditional detectors. We propose \textbf{UniDetector}, a universal object detector that has the ability to recognize enormous categories in the open world. The critical points for the universality of UniDetector are: 1) it leverages images of multiple sources and heterogeneous label spaces for training through the alignment of image and text spaces, which guarantees sufficient information for universal representations. 2) it generalizes to the open world easily while keeping the balance between seen and unseen classes, thanks to abundant information from both vision and language modalities. 3) it further promotes the generalization ability to novel categories through our proposed decoupling training manner and probability calibration. These contributions allow UniDetector to detect over 7k categories, the largest measurable category size so far, with only about 500 classes participating in training. Our UniDetector behaves the strong zero-shot generalization ability on large-vocabulary datasets like LVIS, ImageNetBoxes, and VisualGenome - it surpasses the traditional supervised baselines by more than 4\% on average without seeing any corresponding images. On 13 public detection datasets with various scenes, UniDetector also achieves state-of-the-art performance with only a 3\% amount of training data.

翻译：本文正式阐述了通用目标检测问题，旨在检测所有场景并预测所有类别。传统检测器的通用性受到人类标注依赖、有限视觉信息以及开放世界中新类别的严重制约。我们提出UniDetector——一种通用目标检测器，具备识别开放世界中海量类别的能力。UniDetector实现通用性的关键点在于：1）通过图像与文本空间的对齐，融合多源图像与异构标签空间进行训练，从而保证通用表征的充分信息；2）借助视觉与语言模态的丰富信息，在轻松泛化至开放世界的同时，保持对已知与未知类别的平衡；3）通过提出的解耦训练方式与概率校准方法，进一步提升对新类别的泛化能力。这些贡献使UniDetector可在仅约500个类别参与训练的情况下，检测超过7000个类别——这是迄今可测量的最大类别规模。UniDetector在LVIS、ImageNetBoxes和VisualGenome等大词汇量数据集上展现出强大的零样本泛化能力，在未见对应图像时平均超越传统监督基线方法4%以上。在13个涵盖不同场景的公开检测数据集上，UniDetector仅使用3%的训练数据量即达到最先进性能。