Standard semantic segmentation models owe their success to curated datasets with a fixed set of semantic categories, without contemplating the possibility of identifying unknown objects from novel categories. Existing methods in outlier detection suffer from a lack of smoothness and objectness in their predictions, due to limitations of the per-pixel classification paradigm. Furthermore, additional training for detecting outliers harms the performance of known classes. In this paper, we explore another paradigm with region-level classification to better segment unknown objects. We show that the object queries in mask classification tend to behave like one \vs all classifiers. Based on this finding, we propose a novel outlier scoring function called RbA by defining the event of being an outlier as being rejected by all known classes. Our extensive experiments show that mask classification improves the performance of the existing outlier detection methods, and the best results are achieved with the proposed RbA. We also propose an objective to optimize RbA using minimal outlier supervision. Further fine-tuning with outliers improves the unknown performance, and unlike previous methods, it does not degrade the inlier performance.

翻译：摘要：标准语义分割模型之所以成功，得益于其依赖具有固定语义类别集合的精选数据集，但这类模型未考虑识别来自新类别的未知物体的可能性。现有异常检测方法受限于逐像素分类范式，导致预测结果缺乏平滑性和目标完整性。此外，用于检测异常类的额外训练会损害已知类别的性能。本文探索了另一种基于区域级分类的范式，以更优地分割未知物体。我们揭示掩码分类中的物体查询机制倾向于表现为"一对全"分类器。基于此发现，我们提出名为RbA的新型异常评分函数，其核心思想是将"被所有已知类别拒绝"定义为异常事件。大量实验表明，掩码分类可提升现有异常检测方法的性能，而采用所提出的RbA能取得最佳效果。我们还提出了一种利用最小异常监督优化RbA的目标函数。相较于现有方法，基于异常类的微调不仅能提升未知类性能，且不会降低已知类准确率。