Predictions of opaque black-box systems are frequently deployed in high-stakes applications such as healthcare. For such applications, it is crucial to assess how models handle samples beyond the domain of training data. While several metrics and tests exist to detect out-of-distribution (OoD) data from in-distribution (InD) data to a deep neural network (DNN), their performance varies significantly across datasets, models, and tasks, which limits their practical use. In this paper, we propose a hypothesis-driven approach to quantify whether a new sample is InD or OoD. Given a trained DNN and some input, we first feed the input through the DNN and compute an ensemble of OoD metrics, which we term latent responses. We then formulate the OoD detection problem as a hypothesis test between latent responses of different groups, and use permutation-based resampling to infer the significance of the observed latent responses under a null hypothesis. We adapt our method to detect an unseen sample of bacteria to a trained deep learning model, and show that it reveals interpretable differences between InD and OoD latent responses. Our work has implications for systematic novelty detection and informed decision-making from classifiers trained on a subset of labels.

翻译：不透明黑箱系统的预测结果常被部署在医疗等高敏感性应用中。在此类应用中，评估模型如何处理超出训练数据域的样本至关重要。尽管已有多种指标和测试方法可用于检测深度神经网络（DNN）相对于分布内（InD）数据的分布外（OoD）数据，但这些方法在不同数据集、模型和任务中的性能差异显著，从而限制了其实际应用。本文提出一种基于假设驱动的方法来量化新样本属于InD还是OoD。给定训练好的DNN和输入样本，我们首先将输入馈入DNN，计算一组OoD指标的集成结果，并将其称为潜在响应。随后，我们将OoD检测问题形式化为不同组别潜在响应之间的假设检验，并采用基于置换的重采样方法推断在原假设下观测到的潜在响应显著性。我们将该方法应用于检测训练深度学习模型未见的细菌样本，实验表明其揭示了InD与OoD潜在响应之间可解释的差异。本研究对基于部分标签训练的分类器实现系统性新颖性检测和知情决策具有重要启示。