Explaining predictions of black-box neural networks is crucial when applied to decision-critical tasks. Thus, attribution maps are commonly used to identify important image regions, despite prior work showing that humans prefer explanations based on similar examples. To this end, ProtoPNet learns a set of class-representative feature vectors (prototypes) for case-based reasoning. During inference, similarities of latent features to prototypes are linearly classified to form predictions and attribution maps are provided to explain the similarity. In this work, we evaluate whether architectures for case-based reasoning fulfill established axioms required for faithful explanations using the example of ProtoPNet. We show that such architectures allow the extraction of faithful explanations. However, we prove that the attribution maps used to explain the similarities violate the axioms. We propose a new procedure to extract explanations for trained ProtoPNets, named ProtoPFaith. Conceptually, these explanations are Shapley values, calculated on the similarity scores of each prototype. They allow to faithfully answer which prototypes are present in an unseen image and quantify each pixel's contribution to that presence, thereby complying with all axioms. The theoretical violations of ProtoPNet manifest in our experiments on three datasets (CUB-200-2011, Stanford Dogs, RSNA) and five architectures (ConvNet, ResNet, ResNet50, WideResNet50, ResNeXt50). Our experiments show a qualitative difference between the explanations given by ProtoPNet and ProtoPFaith. Additionally, we quantify the explanations with the Area Over the Perturbation Curve, on which ProtoPFaith outperforms ProtoPNet on all experiments by a factor $>10^3$.

翻译：解释黑盒神经网络的预测在决策关键任务中至关重要。因此，尽管先前研究表明人类更倾向于基于相似示例的解释，但归因图仍常用于识别重要图像区域。为此，ProtoPNet学习一组类别代表性的特征向量（原型）以进行基于案例的推理。在推理过程中，潜在特征与原型的相似度通过线性分类形成预测，并提供归因图来解释相似性。在本工作中，我们以ProtoPNet为例，评估基于案例推理的架构是否满足忠实解释所需的确立公理。我们证明此类架构允许提取忠实解释。然而，我们证实用于解释相似性的归因图违反了公理。我们提出一种名为ProtoPFaith的新方法，用于在训练后的ProtoPNet中提取解释。从概念上讲，这些解释是计算每个原型相似度得分的沙普利值。它们能够忠实回答未见图像中存在哪些原型，并量化每个像素对该存在的贡献，从而满足所有公理。ProtoPNet的理论缺陷在三个数据集（CUB-200-2011、Stanford Dogs、RSNA）和五种架构（ConvNet、ResNet、ResNet50、WideResNet50、ResNeXt50）的实验中得以体现。我们的实验表明，ProtoPNet和ProtoPFaith给出的解释存在定性差异。此外，我们使用扰动曲线下面积量化解释，在所有实验中ProtoPFaith比ProtoPNet高出$>10^3$倍。