Deep neural networks have seen enormous success in various real-world applications. Beyond their predictions as point estimates, increasing attention has been focused on quantifying the uncertainty of their predictions. In this review, we show that the uncertainty of deep neural networks is not only important in a sense of interpretability and transparency, but also crucial in further advancing their performance, particularly in learning systems seeking robustness and efficiency. We will generalize the definition of the uncertainty of deep neural networks to any number or vector that is associated with an input or an input-label pair, and catalog existing methods on ``mining'' such uncertainty from a deep model. We will include those methods from the classic field of uncertainty quantification as well as those methods that are specific to deep neural networks. We then show a wide spectrum of applications of such generalized uncertainty in realistic learning tasks including robust learning such as noisy learning, adversarially robust learning; data-efficient learning such as semi-supervised and weakly-supervised learning; and model-efficient learning such as model compression and knowledge distillation.

翻译：深度神经网络在各类实际应用场景中取得了巨大成功。除了作为点估计的预测结果外，量化其预测的不确定性正日益受到关注。本综述表明，深度神经网络的不确定性不仅对可解释性与透明度具有重要意义，更在进一步提升模型性能——特别是追求鲁棒性与效率的学习系统中——发挥着关键作用。我们将深度神经网络不确定性的定义泛化为与输入或输入-标签对相关联的任意数值或向量，并对从深度模型中"挖掘"此类不确定性的现有方法进行系统分类。这些方法既涵盖经典的不确定性量化领域，也包括专门针对深度神经网络的技术。随后，我们展示了这种广义不确定性在现实学习任务中的广泛应用，包括鲁棒学习（如噪声学习、对抗鲁棒学习）、数据高效学习（如半监督学习与弱监督学习）以及模型高效学习（如模型压缩与知识蒸馏）。