Uncertainty quantification is at the core of the reliability and robustness of machine learning. In this paper, we provide a theoretical framework to dissect the uncertainty, especially the \textit{epistemic} component, in deep learning into \textit{procedural variability} (from the training procedure) and \textit{data variability} (from the training data), which is the first such attempt in the literature to our best knowledge. We then propose two approaches to estimate these uncertainties, one based on influence function and one on batching. We demonstrate how our approaches overcome the computational difficulties in applying classical statistical methods. Experimental evaluations on multiple problem settings corroborate our theory and illustrate how our framework and estimation can provide direct guidance on modeling and data collection efforts.

翻译：不确定性量化是机器学习可靠性与鲁棒性的核心。本文提出一个理论框架，将深度学习中的不确定性（尤其是认知不确定性）分解为**过程变异性**（来自训练过程）和**数据变异性**（来自训练数据），据我们所知，这是文献中首次尝试此类分解。我们进而提出两种估计这些不确定性的方法：一种基于影响函数，另一种基于批处理。我们展示了这些方法如何克服经典统计方法在应用中的计算困难。在多个问题场景上的实验评估验证了我们的理论，并阐明了我们的框架与估计如何为建模与数据收集工作提供直接指导。