Out-of-distribution (OOD) detection plays a crucial role in ensuring the safe deployment of deep neural network (DNN) classifiers. While a myriad of methods have focused on improving the performance of OOD detectors, a critical gap remains in interpreting their decisions. We help bridge this gap by providing explanations for OOD detectors based on learned high-level concepts. We first propose two new metrics for assessing the effectiveness of a particular set of concepts for explaining OOD detectors: 1) detection completeness, which quantifies the sufficiency of concepts for explaining an OOD-detector's decisions, and 2) concept separability, which captures the distributional separation between in-distribution and OOD data in the concept space. Based on these metrics, we propose a framework for learning a set of concepts that satisfy the desired properties of detection completeness and concept separability and demonstrate the framework's effectiveness in providing concept-based explanations for diverse OOD techniques. We also show how to identify prominent concepts that contribute to the detection results via a modified Shapley value-based importance score.

翻译：分布外检测在确保深度神经网络分类器安全部署中起着关键作用。尽管已有大量方法致力于提升分布外检测器的性能，但在解释其决策机制方面仍存在关键空白。为弥合这一空白，我们提出基于学习的高层概念对分布外检测器进行解释。首先提出两个新指标以评估特定概念集对解释分布外检测器的有效性：1）检测完备性，量化概念对解释检测器决策的充分程度；2）概念可分离性，刻画分布内数据与分布外数据在概念空间中的分布分离程度。基于这些指标，我们提出一个学习满足检测完备性与概念可分离性双重特性概念集的框架，并证明该框架能为多种分布外检测技术提供基于概念的解释。此外，通过改进的沙普利值重要性评分，我们展示了如何识别对检测结果起关键作用的显著概念。