In the face of uncertainty, the need for probabilistic assessments has long been recognized in the literature on forecasting. In classification, however, comparative evaluation of classifiers often focuses on predictions specifying a single class through the use of simple accuracy measures, which disregard any probabilistic uncertainty quantification. I propose probabilistic top lists as a novel type of prediction in classification, which bridges the gap between single-class predictions and predictive distributions. The probabilistic top list functional is elicitable through the use of strictly consistent evaluation metrics. The proposed evaluation metrics are based on symmetric proper scoring rules and admit comparison of various types of predictions ranging from single-class point predictions to fully specified predictive distributions. The Brier score yields a metric that is particularly well suited for this kind of comparison.

翻译：面对不确定性时，概率评估的必要性在预测文献中早已得到公认。然而在分类任务中，分类器的比较评估常聚焦于指定单一类别的预测，并采用忽视概率不确定性量化的简单准确率指标。本文提出概率性顶层列表作为分类中一种新型预测方式，它弥合了单类别预测与预测分布之间的鸿沟。概率性顶层列表泛函可通过使用严格一致的评估指标实现可激励性。所提出的评估指标基于对称恰当评分规则，能够比较从单类别点预测到完全指定的预测分布等各类预测。其中，布里尔评分特别适合此类比较。