Detecting out-of-distribution (OOD) data is a task that is receiving an increasing amount of research attention in the domain of deep learning for computer vision. However, the performance of detection methods is generally evaluated on the task in isolation, rather than also considering potential downstream tasks in tandem. In this work, we examine selective classification in the presence of OOD data (SCOD). That is to say, the motivation for detecting OOD samples is to reject them so their impact on the quality of predictions is reduced. We show under this task specification, that existing post-hoc methods perform quite differently compared to when evaluated only on OOD detection. This is because it is no longer an issue to conflate in-distribution (ID) data with OOD data if the ID data is going to be misclassified. However, the conflation within ID data of correct and incorrect predictions becomes undesirable. We also propose a novel method for SCOD, Softmax Information Retaining Combination (SIRC), that augments softmax-based confidence scores with feature-agnostic information such that their ability to identify OOD samples is improved without sacrificing separation between correct and incorrect ID predictions. Experiments on a wide variety of ImageNet-scale datasets and convolutional neural network architectures show that SIRC is able to consistently match or outperform the baseline for SCOD, whilst existing OOD detection methods fail to do so.

翻译：检测分布外（OOD）数据是计算机视觉深度学习领域日益受到关注的研究任务。然而，检测方法的性能通常仅在孤立的检测任务上进行评估，而非同时考虑潜在的下游任务。在本研究中，我们探讨了在存在OOD数据情况下的选择性分类（SCOD）。换言之，检测OOD样本的动机在于将其剔除，从而减少其对预测质量的影响。我们证明，在此任务规范下，现有的后处理方法的表现与仅在OOD检测任务上评估时大相径庭。这是因为，如果分布内（ID）数据将被错误分类，那么将ID数据与OOD数据混淆不再是主要问题。然而，在ID数据中，正确预测与错误预测之间的混淆变得不可取。我们还提出了一种用于SCOD的新方法——软最大信息保留组合（SIRC），该方法通过增强基于软最大值的置信度分数，融入与特征无关的信息，从而在不牺牲ID数据中正确与错误预测分离能力的前提下，提升识别OOD样本的能力。在多种ImageNet规模数据集和卷积神经网络架构上的实验表明，SIRC能够持续匹配或超越SCOD的基线方法，而现有的OOD检测方法则无法做到这一点。