Decomposing prediction uncertainty into its aleatoric (irreducible) and epistemic (reducible) components is critical for the development and deployment of machine learning systems. A popular, principled measure for epistemic uncertainty is the mutual information between the response variable and model parameters. However, evaluating this measure requires access to the posterior distribution of the model parameters, which is challenging to compute. In view of this, we introduce a frequentist measure of epistemic uncertainty based on the bootstrap. Our main theoretical contribution is a novel asymptotic expansion that reveals that our proposed (frequentist) measure and the (Bayesian) mutual information are asymptotically equivalent. This provides frequentist interpretations to mutual information and new computational strategies for approximating it. Moreover, we link our proposed approach to the widely-used heuristic approach of deep ensembles, giving added perspective on their practical success.

翻译：将预测不确定性分解为偶然性（不可约）和认知性（可约）分量对于机器学习系统的开发与部署至关重要。一种流行且具有理论依据的认知不确定性度量是响应变量与模型参数之间的互信息。然而，评估该度量需要获取模型参数的后验分布，这在计算上具有挑战性。鉴于此，我们提出一种基于自助法的频率学派认知不确定性度量。我们的主要理论贡献是一个新颖的渐近展开式，揭示了所提出的（频率学派）度量与（贝叶斯）互信息在渐近意义上是等价的。这为互信息提供了频率学派的解释，并提出了近似计算的新策略。此外，我们将所提出的方法与广泛使用的深度集成启发式方法联系起来，为其实际成功提供了新的视角。