Deep neural networks lack interpretability and tend to be overconfident, which poses a serious problem in safety-critical applications like autonomous driving, medical imaging, or machine vision tasks with high demands on reliability. Quantifying the predictive uncertainty is a promising endeavour to open up the use of deep neural networks for such applications. Unfortunately, current available methods are computationally expensive. In this work, we present a novel approach for efficient and reliable uncertainty estimation which we call Deep Uncertainty Distillation using Ensembles for Segmentation (DUDES). DUDES applies student-teacher distillation with a Deep Ensemble to accurately approximate predictive uncertainties with a single forward pass while maintaining simplicity and adaptability. Experimentally, DUDES accurately captures predictive uncertainties without sacrificing performance on the segmentation task and indicates impressive capabilities of identifying wrongly classified pixels and out-of-domain samples on the Cityscapes dataset. With DUDES, we manage to simultaneously simplify and outperform previous work on Deep Ensemble-based Uncertainty Distillation.

翻译：深度神经网络缺乏可解释性且倾向于过度自信，这在自动驾驶、医学影像或对可靠性要求极高的机器视觉任务等安全关键应用中引发了严重问题。量化预测不确定性是推动深度神经网络应用于此类场景的重要方向。然而，现有方法通常计算成本高昂。本文提出一种高效且可靠的不确定性估计新方法，称为“利用集成进行深度不确定性蒸馏的语义分割”（DUDES）。该方法通过深度集成进行师生蒸馏，在保持简洁性和适应性的同时，仅需单次前向传播即可精确近似预测不确定性。实验表明，DUDES在保持分割任务性能不下降的前提下精准捕捉预测不确定性，并在Cityscapes数据集上展现出识别误分类像素与域外样本的卓越能力。通过DUDES，我们成功简化并超越了以往基于深度集成的不确定性蒸馏研究工作。