The evaluation of the fidelity of eXplainable Artificial Intelligence (XAI) methods to their underlying models is a challenging task, primarily due to the absence of a ground truth for explanations. However, assessing fidelity is a necessary step for ensuring a correct XAI methodology. In this study, we conduct a fair and objective comparison of the current state-of-the-art XAI methods by introducing three novel image datasets with reliable ground truth for explanations. The primary objective of this comparison is to identify methods with low fidelity and eliminate them from further research, thereby promoting the development of more trustworthy and effective XAI techniques. Our results demonstrate that XAI methods based on the backpropagation of output information to input yield higher accuracy and reliability compared to methods relying on sensitivity analysis or Class Activation Maps (CAM). However, the backpropagation method tends to generate more noisy saliency maps. These findings have significant implications for the advancement of XAI methods, enabling the elimination of erroneous explanations and fostering the development of more robust and reliable XAI.

翻译：评估可解释人工智能（XAI）方法对其底层模型的忠实度是一项具有挑战性的任务，主要原因是缺乏解释的真实基准。然而，评估忠实度是确保正确XAI方法论的必要步骤。在本研究中，我们通过引入三个具有可靠解释真实基准的新型图像数据集，对当前最先进的XAI方法进行了公平客观的比较。该比较的主要目标是识别出忠实度较低的方法并将其从进一步研究中剔除，从而促进更可信、更有效的XAI技术的发展。我们的结果表明，基于将输出信息反向传播至输入的XAI方法相比依赖敏感性分析或类激活映射（CAM）的方法，具有更高的准确性和可靠性。然而，反向传播方法倾向于生成噪声更大的显著性图。这些发现对XAI方法的进步具有重要意义，能够消除错误解释并推动更稳健、更可靠的XAI技术的发展。