Robustness to out-of-distribution (OOD) samples is crucial for safely deploying machine learning models in the open world. Recent works have focused on designing scoring functions to quantify OOD uncertainty. Setting appropriate thresholds for these scoring functions for OOD detection is challenging as OOD samples are often unavailable up front. Typically, thresholds are set to achieve a desired true positive rate (TPR), e.g., $95\%$ TPR. However, this can lead to very high false positive rates (FPR), ranging from 60 to 96\%, as observed in the Open-OOD benchmark. In safety-critical real-life applications, e.g., medical diagnosis, controlling the FPR is essential when dealing with various OOD samples dynamically. To address these challenges, we propose a mathematically grounded OOD detection framework that leverages expert feedback to \emph{safely} update the threshold on the fly. We provide theoretical results showing that it is guaranteed to meet the FPR constraint at all times while minimizing the use of human feedback. Another key feature of our framework is that it can work with any scoring function for OOD uncertainty quantification. Empirical evaluation of our system on synthetic and benchmark OOD datasets shows that our method can maintain FPR at most $5\%$ while maximizing TPR.

翻译：分布外样本的鲁棒性对于在开放世界中安全部署机器学习模型至关重要。近期研究聚焦于设计评分函数以量化分布外不确定性。由于分布外样本通常无法预先获取，为这些评分函数设定合适的检测阈值颇具挑战。通常，阈值设定旨在实现期望的真阳性率（如$95\%$ TPR），但Open-OOD基准测试表明，这可能导致高达60%至96%的极高假阳性率。在医疗诊断等安全关键型现实应用中，动态处理各类分布外样本时，控制FPR至关重要。为解决上述挑战，我们提出一种基于数学理论的分布外检测框架，该框架利用专家反馈动态安全调整阈值。理论证明，该框架可在最小化人类反馈使用量的同时，始终满足FPR约束条件。另一关键特性是，本框架兼容任意分布外不确定性量化评分函数。在合成数据集与基准OOD数据集上的实证评估表明，本方法在最大化TPR的同时，可将FPR维持在$5\%$以下。