Uncertainty representation and quantification are paramount in machine learning and constitute an important prerequisite for safety-critical applications. In this paper, we propose novel measures for the quantification of aleatoric and epistemic uncertainty based on proper scoring rules, which are loss functions with the meaningful property that they incentivize the learner to predict ground-truth (conditional) probabilities. We assume two common representations of (epistemic) uncertainty, namely, in terms of a credal set, i.e. a set of probability distributions, or a second-order distribution, i.e., a distribution over probability distributions. Our framework establishes a natural bridge between these representations. We provide a formal justification of our approach and introduce new measures of epistemic and aleatoric uncertainty as concrete instantiations.

翻译：不确定性表征与量化是机器学习领域的核心问题，亦是安全关键应用的重要前提。本文基于适当评分规则提出新型随机不确定性与认知不确定性量化度量方法。适当评分规则作为损失函数具备重要性质：能够激励学习器预测真实（条件）概率。假设认知不确定性的两种常见表征形式——即信度集（概率分布集合）与二阶分布（概率分布上的分布），本研究框架在这两种表征形式间建立了天然桥梁。我们为所提方法提供形式化理论依据，并引入具体实例化的新型认知与随机不确定性度量。