We study the problem of Out-of-Distribution (OOD) detection, that is, detecting whether a learning algorithm's output can be trusted at inference time. While a number of tests for OOD detection have been proposed in prior work, a formal framework for studying this problem is lacking. We propose a definition for the notion of OOD that includes both the input distribution and the learning algorithm, which provides insights for the construction of powerful tests for OOD detection. We propose a multiple hypothesis testing inspired procedure to systematically combine any number of different statistics from the learning algorithm using conformal p-values. We further provide strong guarantees on the probability of incorrectly classifying an in-distribution sample as OOD. In our experiments, we find that threshold-based tests proposed in prior work perform well in specific settings, but not uniformly well across different types of OOD instances. In contrast, our proposed method that combines multiple statistics performs uniformly well across different datasets and neural networks.

翻译：我们研究了分布外检测问题，即在推理阶段判断学习算法的输出是否可信。尽管已有大量关于分布外检测的统计检验方法被提出，但目前仍缺乏对此问题的系统性研究框架。我们提出了一种同时包含输入分布与学习算法的分布外定义，这为构建强有力的分布外检测方法提供了理论支撑。我们设计了一种基于多重假设检验的流程，利用共形p值系统性地整合学习算法产生的任意多个统计量。该方法对误将分布内样本分类为分布外样本的概率提供了严格的理论保证。实验表明，现有基于阈值的检测方法在特定场景表现良好，但无法统一适用于不同类型的分布外样本。相比之下，我们提出的多统计量融合方法在不同数据集和神经网络架构上均展现出稳定优越的性能。