In this work, we perform an in-depth analysis of the visualisation methods implemented in two popular self-explaining models for visual classification based on prototypes - ProtoPNet and ProtoTree. Using two fine-grained datasets (CUB-200-2011 and Stanford Cars), we first show that such methods do not correctly identify the regions of interest inside of the images, and therefore do not reflect the model behaviour. Secondly, using a deletion metric, we demonstrate quantitatively that saliency methods such as Smoothgrads or PRP provide more faithful image patches. We also propose a new relevance metric based on the segmentation of the object provided in some datasets (e.g. CUB-200-2011) and show that the imprecise patch visualisations generated by ProtoPNet and ProtoTree can create a false sense of bias that can be mitigated by the use of more faithful methods. Finally, we discuss the implications of our findings for other prototype-based models sharing the same visualisation method.

翻译：本文对两种流行的基于原型的视觉分类自解释模型——ProtoPNet和ProtoTree——中实现的可视化方法进行了深入分析。通过使用两个细粒度数据集（CUB-200-2011和Stanford Cars），我们首先证明这些方法无法正确识别图像中的感兴趣区域，因此不能反映模型行为。其次，利用删除度量指标，我们定量证明了Smoothgrads或PRP等显著性方法能提供更忠实的图像补丁。我们还提出了一种基于某些数据集（如CUB-200-2011）中提供的对象分割的新相关性度量，并表明ProtoPNet和ProtoTree生成的不精确补丁可视化可能产生错误的偏差感，而使用更忠实的方法可以缓解这一问题。最后，我们讨论了这些发现对共享相同可视化方法的其他基于原型的模型的影响。