For debugging and verification of computer vision convolutional deep neural networks (CNNs) human inspection of the learned latent representations is imperative. Therefore, state-of-the-art eXplainable Artificial Intelligence (XAI) methods globally associate given natural language semantic concepts with representing vectors or regions in the CNN latent space supporting manual inspection. Yet, this approach comes with two major disadvantages: They are locally inaccurate when reconstructing a concept label and discard information about the distribution of concept instance representations. The latter, though, is of particular interest for debugging, like finding and understanding outliers, learned notions of sub-concepts, and concept confusion. Furthermore, current single-layer approaches neglect that information about a concept may be spread over the CNN depth. To overcome these shortcomings, we introduce the local-to-global Guided Concept Projection Vectors (GCPV) approach: It (1) generates local concept vectors that each precisely reconstruct a concept segmentation label, and then (2) generalizes these to global concept and even sub-concept vectors by means of hiearchical clustering. Our experiments on object detectors demonstrate improved performance compared to the state-of-the-art, the benefit of multi-layer concept vectors, and robustness against low-quality concept segmentation labels. Finally, we demonstrate that GCPVs can be applied to find root causes for confusion of concepts like bus and truck, and reveal interesting concept-level outliers. Thus, GCPVs pose a promising step towards interpretable model debugging and informed data improvement.

翻译：为调试和验证计算机视觉卷积深度神经网络（CNN），人工检查所学到的潜在表征至关重要。因此，最先进的可解释人工智能（XAI）方法将给定的自然语言语义概念与CNN潜在空间中代表向量或区域全局关联，以支持人工检查。然而，这种方法存在两大缺陷：在重构概念标签时局部不准确，且丢弃了关于概念实例表征分布的信息。后者对于调试（例如发现并理解异常值、所学子概念以及概念混淆）具有重要价值。此外，当前的单层方法忽视了概念信息可能分布于CNN整个深度这一事实。为克服这些不足，我们提出了局部到全局的引导概念投影向量（GCPV）方法：该方法（1）生成局部概念向量，每个向量能精确重构概念分割标签，然后（2）通过层次聚类将这些向量泛化为全局概念向量乃至子概念向量。我们在目标检测器上的实验表明，与最先进方法相比，GCPV性能更优，具备多层概念向量的优势，且对低质量概念分割标签具有鲁棒性。最后，我们展示了GCPV可应用于发现公交与卡车等概念混淆的根本原因，并揭示有趣的概念级异常值。因此，GCPV为实现可解释模型调试和智能数据改进迈出了有前景的一步。