Deep neural networks (DNNs) have achieved tremendous success in making accurate predictions for computer vision, natural language processing, as well as science and engineering domains. However, it is also well-recognized that DNNs sometimes make unexpected, incorrect, but overconfident predictions. This can cause serious consequences in high-stake applications, such as autonomous driving, medical diagnosis, and disaster response. Uncertainty quantification (UQ) aims to estimate the confidence of DNN predictions beyond prediction accuracy. In recent years, many UQ methods have been developed for DNNs. It is of great practical value to systematically categorize these UQ methods and compare their advantages and disadvantages. However, existing surveys mostly focus on categorizing UQ methodologies from a neural network architecture perspective or a Bayesian perspective and ignore the source of uncertainty that each methodology can incorporate, making it difficult to select an appropriate UQ method in practice. To fill the gap, this paper presents a systematic taxonomy of UQ methods for DNNs based on the types of uncertainty sources (data uncertainty versus model uncertainty). We summarize the advantages and disadvantages of methods in each category. We show how our taxonomy of UQ methodologies can potentially help guide the choice of UQ method in different machine learning problems (e.g., active learning, robustness, and reinforcement learning). We also identify current research gaps and propose several future research directions.

翻译：深度神经网络在计算机视觉、自然语言处理以及科学与工程领域取得了巨大成功，能够做出精确预测。然而，人们也普遍认识到，深度神经网络有时会做出意外、错误但过于自信的预测。这在自动驾驶、医学诊断和灾害响应等高风险应用中可能导致严重后果。不确定性量化旨在估计深度神经网络预测的置信度，超越预测精度的范畴。近年来，针对深度神经网络开发了许多不确定性量化方法。系统性地对这些方法进行分类并比较其优缺点具有重要的实用价值。然而，现有综述大多从神经网络架构视角或贝叶斯视角对不确定性量化方法进行分类，忽略了每种方法能够整合的不确定性来源，这使得在实践中难以选择合适的不确定性量化方法。为填补这一空白，本文基于不确定性来源类型（数据不确定性对比模型不确定性），提出了深度神经网络不确定性量化方法的系统分类法。我们总结了每个类别中方法的优缺点，展示了我们的不确定性量化方法论分类如何帮助指导不同机器学习问题（如主动学习、鲁棒性和强化学习）中不确定性量化方法的选择。我们还指出了当前的研究空白，并提出了若干未来研究方向。