Reliable application of machine learning-based decision systems in the wild is one of the major challenges currently investigated by the field. A large portion of established approaches aims to detect erroneous predictions by means of assigning confidence scores. This confidence may be obtained by either quantifying the model's predictive uncertainty, learning explicit scoring functions, or assessing whether the input is in line with the training distribution. Curiously, while these approaches all state to address the same eventual goal of detecting failures of a classifier upon real-life application, they currently constitute largely separated research fields with individual evaluation protocols, which either exclude a substantial part of relevant methods or ignore large parts of relevant failure sources. In this work, we systematically reveal current pitfalls caused by these inconsistencies and derive requirements for a holistic and realistic evaluation of failure detection. To demonstrate the relevance of this unified perspective, we present a large-scale empirical study for the first time enabling benchmarking confidence scoring functions w.r.t all relevant methods and failure sources. The revelation of a simple softmax response baseline as the overall best performing method underlines the drastic shortcomings of current evaluation in the abundance of publicized research on confidence scoring. Code and trained models are at https://github.com/IML-DKFZ/fd-shifts.

翻译：在现实世界中可靠应用基于机器学习决策系统是当前研究领域面临的主要挑战之一。大量已有方法旨在通过分配置信度分数来检测错误预测。这种置信度可以通过量化模型预测不确定性、学习显式评分函数，或评估输入是否与训练分布一致来获得。有趣的是，尽管这些方法都声称旨在解决在真实应用中检测分类器故障的相同最终目标，但它们目前构成了各自独立的、具有不同评估协议的研究领域。这些协议要么排除了大部分相关方法，要么忽略了大部分相关的故障来源。在本工作中，我们系统揭示了这些不一致性导致的当前缺陷，并推导出对故障检测进行全面且现实评估的要求。为展示这一统一视角的相关性，我们首次进行了一项大规模实证研究，能够对置信度评分函数在所有相关方法和故障来源上进行基准测试。一个简单的Softmax响应基线作为整体最佳性能方法的发现，凸显了当前评估在大量关于置信度评分的已发表研究中的严重不足。代码和训练模型位于https://github.com/IML-DKFZ/fd-shifts。