We study the problem of Out-of-Distribution (OOD) detection, that is, detecting whether a learning algorithm's output can be trusted at inference time. While a number of tests for OOD detection have been proposed in prior work, a formal framework for studying this problem is lacking. We propose a definition for the notion of OOD that includes both the input distribution and the learning algorithm, which provides insights for the construction of powerful tests for OOD detection. We propose a multiple hypothesis testing inspired procedure to systematically combine any number of different statistics from the learning algorithm using conformal p-values. We further provide strong guarantees on the probability of incorrectly classifying an in-distribution sample as OOD. In our experiments, we find that threshold-based tests proposed in prior work perform well in specific settings, but not uniformly well across different types of OOD instances. In contrast, our proposed method that combines multiple statistics performs uniformly well across different datasets and neural networks.

翻译：我们研究分布外（OOD）检测问题，即判断学习算法在推理阶段的输出是否可信。尽管已有研究提出多种OOD检测方法，但目前仍缺乏系统的形式化理论框架。我们提出了包含输入分布与学习算法的OOD概念定义，这为构建高性能OOD检测方法提供了理论指导。本文基于多重假设检验思想，提出通过共形p值系统整合学习算法中任意数量统计量的新方法。我们进一步给出了将分布内样本误判为OOD的概率严格保证。实验表明，现有基于阈值的检测方法在特定场景下表现良好，但无法在不同类型OOD实例中保持稳定性能。相比之下，本文提出的多统计量融合方法在不同数据集和神经网络架构上均表现出鲁棒性。