Understanding when and why deep neural networks are uncertain is crucial for deploying reliable machine learning systems in safety-critical domains. While existing uncertainty quantification methods provide scalar measures of model confidence, they offer limited insight into which spatial regions of an input contribute to different types of uncertainty. We propose a novel visualization framework, Uncertainty Activation Map (UAM), that combines Evidential Deep Learning (EDL) with Full-Gradient Class Activation Mapping (FullGrad) to generate interpretable spatial uncertainty activation maps. Our approach distinguishes between two fundamental types of uncertainty: vacuity, representing lack of evidence, and dissonance, capturing conflicting evidence between competing hypotheses. By leveraging the complete gradient decomposition property of FullGrad and the principled uncertainty quantification of Subjective Logic, our method produces theoretically grounded visualizations that highlight specific image regions responsible for model uncertainty. With this framework, vacuity and dissonance activation maps are generated by computing belief-weighted attributions, enabling identification of where models lack knowledge versus where they encounter ambiguous evidence. Extensive evaluations across multiple benchmark datasets demonstrate that the proposed framework effectively addresses the critical gap between uncertainty quantification and explainability, providing intuitive visual feedback to assess model reliability in complex visual recognition tasks.
翻译:理解深度神经网络何时以及为何存在不确定性,对于在安全关键领域部署可靠的机器学习系统至关重要。尽管现有不确定性量化方法提供了模型置信度的标量度量,但它们对于输入中哪些空间区域贡献于不同类型的不确定性提供的见解有限。我们提出了一种新颖的可视化框架——不确定性激活图(UAM),该框架将证据深度学习(EDL)与全梯度类激活映射(FullGrad)相结合,以生成可解释的空间不确定性激活图。我们的方法区分了两种基本的不确定性类型:空源性(vacuity),代表缺乏证据,以及失调性(dissonance),捕捉竞争假设之间的冲突证据。通过利用FullGrad的完整梯度分解特性以及主观逻辑的原则性不确定性量化,我们的方法生成了具有理论基础的视觉化结果,突出了导致模型不确定性的特定图像区域。借助该框架,通过计算基于信仰的归因生成空源性和失调性激活图,从而能够识别模型缺乏知识的区域与遇到模糊证据的区域之间的区别。在多个基准数据集上的广泛评估表明,所提出的框架有效解决了不确定性量化与可解释性之间的关键鸿沟,为评估复杂视觉识别任务中的模型可靠性提供了直观的视觉反馈。