While deep neural network models offer unmatched classification performance, they are prone to learning spurious correlations in the data. Such dependencies on confounding information can be difficult to detect using performance metrics if the test data comes from the same distribution as the training data. Interpretable ML methods such as post-hoc explanations or inherently interpretable classifiers promise to identify faulty model reasoning. However, there is mixed evidence whether many of these techniques are actually able to do so. In this paper, we propose a rigorous evaluation strategy to assess an explanation technique's ability to correctly identify spurious correlations. Using this strategy, we evaluate five post-hoc explanation techniques and one inherently interpretable method for their ability to detect three types of artificially added confounders in a chest x-ray diagnosis task. We find that the post-hoc technique SHAP, as well as the inherently interpretable Attri-Net provide the best performance and can be used to reliably identify faulty model behavior.

翻译：尽管深度神经网络模型在分类任务中展现了无与伦比的性能，但它们容易学习数据中的虚假相关。当测试数据与训练数据同分布时，这种对混淆信息的依赖难以通过性能指标检测。可解释机器学习方法（如事后解释或本质可解释分类器）有望识别出有缺陷的模型推理过程。然而，关于这些技术实际是否具备此能力，现有证据仍存在争议。本文提出了一种严格的评估策略，用于检验解释技术正确识别虚假相关的能力。基于该策略，我们评估了五种事后解释技术和一种本质可解释方法在胸部X光诊断任务中检测三类人工添加混淆因素的能力。结果表明，事后解释技术SHAP与本质可解释方法Attri-Net表现最佳，可可靠地识别模型的错误行为。