Deep neural networks (DNNs) have achieved tremendous success in making accurate predictions for computer vision, natural language processing, as well as science and engineering domains. However, it is also well-recognized that DNNs sometimes make unexpected, incorrect, but overconfident predictions. This can cause serious consequences in high-stake applications, such as autonomous driving, medical diagnosis, and disaster response. Uncertainty quantification (UQ) aims to estimate the confidence of DNN predictions beyond prediction accuracy. In recent years, many UQ methods have been developed for DNNs. It is of great practical value to systematically categorize these UQ methods and compare their advantages and disadvantages. However, existing surveys mostly focus on categorizing UQ methodologies from a neural network architecture perspective or a Bayesian perspective and ignore the source of uncertainty that each methodology can incorporate, making it difficult to select an appropriate UQ method in practice. To fill the gap, this paper presents a systematic taxonomy of UQ methods for DNNs based on the types of uncertainty sources (data uncertainty versus model uncertainty). We summarize the advantages and disadvantages of methods in each category. We show how our taxonomy of UQ methodologies can potentially help guide the choice of UQ method in different machine learning problems (e.g., active learning, robustness, and reinforcement learning). We also identify current research gaps and propose several future research directions.

翻译：深度神经网络（DNNs）在计算机视觉、自然语言处理以及科学与工程领域取得了巨大成功，能够实现精确预测。然而，人们也普遍认识到，DNNs有时会做出意外、错误但过度自信的预测。这在自动驾驶、医疗诊断和灾害响应等高风险应用中可能造成严重后果。不确定性量化（UQ）旨在超越预测精度，评估DNN预测的置信度。近年来，针对DNNs已开发出许多UQ方法。系统性地对这些UQ方法进行分类并比较其优缺点具有重要的实际价值。然而，现有综述大多从神经网络架构视角或贝叶斯视角对UQ方法进行归类，忽略了每种方法所能融合的不确定性源，导致实践中难以选择合适的UQ方法。为填补这一空白，本文基于不确定性源类型（数据不确定性与模型不确定性），提出了一种针对DNNs的UQ方法系统分类法。我们总结了每类方法的优缺点，并展示了基于不确定性源的UQ方法分类如何有助于在不同机器学习问题（如主动学习、鲁棒性和强化学习）中指导UQ方法的选择。此外，我们还指出了当前研究空白，并提出了若干未来研究方向。