High-performing out-of-distribution (OOD) detection, both anomaly and novel class, is an important prerequisite for the practical use of classification models. In this paper, we focus on the species recognition task in images concerned with large databases, a large number of fine-grained hierarchical classes, severe class imbalance, and varying image quality. We propose a framework for combining individual OOD measures into one combined OOD (COOD) measure using a supervised model. The individual measures are several existing state-of-the-art measures and several novel OOD measures developed with novel class detection and hierarchical class structure in mind. COOD was extensively evaluated on three large-scale (500k+ images) biodiversity datasets in the context of anomaly and novel class detection. We show that COOD outperforms individual, including state-of-the-art, OOD measures by a large margin in terms of TPR@1% FPR in the majority of experiments, e.g., improving detecting ImageNet images (OOD) from 54.3% to 85.4% for the iNaturalist 2018 dataset. SHAP (feature contribution) analysis shows that different individual OOD measures are essential for various tasks, indicating that multiple OOD measures and combinations are needed to generalize. Additionally, we show that explicitly considering ID images that are incorrectly classified for the original (species) recognition task is important for constructing high-performing OOD detection methods and for practical applicability. The framework can easily be extended or adapted to other tasks and media modalities.

翻译：摘要： 高性能的分布外（OOD）检测（包括异常检测与新型类别检测）是分类模型实际应用的重要前提。本文聚焦于图像物种识别任务，该任务涉及大规模数据库、大量细粒度层级类别、严重的类别不平衡以及多变的图像质量。我们提出一个框架，通过监督模型将多种独立OOD度量整合为组合式分布外（COOD）度量。这些独立度量包括若干现有最优方法以及为新型类别检测和层级类别结构而开发的新型OOD度量。在异常与新型类别检测背景下，我们在三个大规模（50万+图像）生物多样性数据集上对COOD进行了全面评估。实验表明，在多数实验中，COOD在1%假阳性率下的真阳性率（TPR@1% FPR）指标上大幅优于包括现有最优方法在内的独立OOD度量——例如，针对iNaturalist 2018数据集，将ImageNet图像（OOD）的检测率从54.3%提升至85.4%。SHAP（特征贡献）分析表明，不同独立OOD度量对各类任务至关重要，说明泛化需要多种OOD度量及其组合。此外，我们证实：显式考虑原始（物种）识别任务中错误分类的分布内（ID）图像，对于构建高性能OOD检测方法及其实际应用具有重要意义。该框架可便捷扩展或适配至其他任务与媒介模态。