Aleatoric (data) and epistemic (knowledge) uncertainty are textbook components of Uncertainty Quantification. Jointly estimating these components has been shown to be problematic and non-trivial. As a result, there are multiple ways to disentangle these uncertainties, but current methods to evaluate them are insufficient. We propose that aleatoric and epistemic uncertainty estimates should be orthogonally disentangled - meaning that each uncertainty is not affected by the other - a necessary condition that is often not met. We prove that orthogonality and consistency and necessary and sufficient criteria for disentanglement, and construct Uncertainty Disentanglement Error as a metric to measure these criteria, with further empirical evaluation showing that finetuned models give different orthogonality results than models trained from scratch and that UDE can be optimized for through dropout rate. We demonstrate a Deep Ensemble trained from scratch on ImageNet-1k with Information Theoretic disentangling achieves consistent and orthogonal estimates of epistemic uncertainty, but estimates of aleatoric uncertainty still fail on orthogonality.

翻译：偶然性（数据）不确定性和认知性（知识）不确定性是不确定性量化中的经典组成部分。联合估计这些成分已被证明存在问题且非易事。因此，存在多种解耦这些不确定性的方法，但目前评估这些方法的手段尚不充分。我们提出，偶然性和认知性不确定性估计应被正交解耦——即每种不确定性不受另一种影响——这是一个常未满足的必要条件。我们证明正交性和一致性是解耦的充分必要条件，并构建了不确定性解耦误差作为度量这些准则的指标，进一步的实证评估表明，微调模型给出的正交性结果与从头训练的模型不同，且不确定性解耦误差可通过丢弃率进行优化。我们证明，在ImageNet-1k上使用信息论解耦方法从头训练的深度集成模型能获得一致且正交的认知性不确定性估计，但偶然性不确定性的估计在正交性上仍然失败。