Deep neural networks (DNNs) have achieved tremendous success in making accurate predictions for computer vision, natural language processing, as well as science and engineering domains. However, it is also well-recognized that DNNs sometimes make unexpected, incorrect, but overconfident predictions. This can cause serious consequences in high-stake applications, such as autonomous driving, medical diagnosis, and disaster response. Uncertainty quantification (UQ) aims to estimate the confidence of DNN predictions beyond prediction accuracy. In recent years, many UQ methods have been developed for DNNs. It is of great practical value to systematically categorize these UQ methods and compare their advantages and disadvantages. However, existing surveys mostly focus on categorizing UQ methodologies from a neural network architecture perspective or a Bayesian perspective and ignore the source of uncertainty that each methodology can incorporate, making it difficult to select an appropriate UQ method in practice. To fill the gap, this paper presents a systematic taxonomy of UQ methods for DNNs based on the types of uncertainty sources (data uncertainty versus model uncertainty). We summarize the advantages and disadvantages of methods in each category. We show how our taxonomy of UQ methodologies can potentially help guide the choice of UQ method in different machine learning problems (e.g., active learning, robustness, and reinforcement learning). We also identify current research gaps and propose several future research directions.

翻译：深度神经网络(DNNs)在计算机视觉、自然语言处理以及科学与工程领域已取得巨大成功。然而，人们也已认识到，DNNs有时会做出出乎意料、错误但过于自信的预测。这在自动驾驶、医学诊断和灾难响应等高风险应用中可能引发严重后果。不确定性量化旨在评估DNN预测的置信度，而非仅关注预测精度。近年来，已涌现出大量针对DNNs的不确定性量化方法。系统性地对这些UQ方法进行分类并比较其优劣具有重要的实际价值。然而，现有综述大多从神经网络架构视角或贝叶斯视角对UQ方法进行分类，忽略了每种方法所能融合的不确定性来源，导致实践中难以选择合适的UQ方法。为填补这一空白，本文基于不确定性来源类型（数据不确定性与模型不确定性），提出了针对DNNs的UQ方法系统分类体系。我们总结了各类方法的优缺点，展示了所提出的UQ方法分类体系如何助力指导不同机器学习问题（如主动学习、鲁棒性和强化学习）中的UQ方法选择。此外，我们还指出了当前的研究空白，并提出了若干未来研究方向。