Open-set Recognition (OSR) aims to identify test samples whose classes are not seen during the training process. Recently, Unified Open-set Recognition (UOSR) has been proposed to reject not only unknown samples but also known but wrongly classified samples, which tends to be more practical in real-world applications. The UOSR draws little attention since it is proposed, but we find sometimes it is even more practical than OSR in the real world applications, as evaluation results of known but wrongly classified samples are also wrong like unknown samples. In this paper, we deeply analyze the UOSR task under different training and evaluation settings to shed light on this promising research direction. For this purpose, we first evaluate the UOSR performance of several OSR methods and show a significant finding that the UOSR performance consistently surpasses the OSR performance by a large margin for the same method. We show that the reason lies in the known but wrongly classified samples, as their uncertainty distribution is extremely close to unknown samples rather than known and correctly classified samples. Second, we analyze how the two training settings of OSR (i.e., pre-training and outlier exposure) influence the UOSR. We find although they are both beneficial for distinguishing known and correctly classified samples from unknown samples, pre-training is also helpful for identifying known but wrongly classified samples while outlier exposure is not. In addition to different training settings, we also formulate a new evaluation setting for UOSR which is called few-shot UOSR, where only one or five samples per unknown class are available during evaluation to help identify unknown samples. We propose FS-KNNS for the few-shot UOSR to achieve state-of-the-art performance under all settings.

翻译：开放集识别（OSR）旨在识别训练过程中未见类别的测试样本。近年来提出的统一开放集识别（UOSR）不仅需要拒斥未知样本，还需拒斥已知但被错误分类的样本，这在现实应用中更具实用性。自提出以来，UOSR虽未受广泛关注，但我们发现其在现实应用中的实用性甚至常超越OSR，因为已知但被错误分类样本的评估结果与未知样本同样存在错误。本文深入分析不同训练与评估设置下的UOSR任务，以阐明这一具有前景的研究方向。为此，我们首先评估多种OSR方法的UOSR性能，发现一个显著结论：同一方法下UOSR性能始终大幅超越OSR性能。研究表明，其原因在于已知但被错误分类的样本：其不确定性分布与未知样本极为相似，而非已知且正确分类的样本。其次，我们分析OSR的两类训练设置（即预训练与异常暴露）对UOSR的影响。发现尽管两者均有助于区分已知正确分类样本与未知样本，但预训练还能辅助识别已知错误分类样本，而异常暴露则无法实现。除不同训练设置外，我们还为UOSR制定了新的评估设置——少样本UOSR，即评估时每个未知类别仅提供1或5个样本以辅助识别未知样本。为此提出FS-KNNS方法，在全部设置下均实现最先进性能。