While deep neural network models offer unmatched classification performance, they are prone to learning spurious correlations in the data. Such dependencies on confounding information can be difficult to detect using performance metrics if the test data comes from the same distribution as the training data. Interpretable ML methods such as post-hoc explanations or inherently interpretable classifiers promise to identify faulty model reasoning. However, there is mixed evidence whether many of these techniques are actually able to do so. In this paper, we propose a rigorous evaluation strategy to assess an explanation technique's ability to correctly identify spurious correlations. Using this strategy, we evaluate five post-hoc explanation techniques and one inherently interpretable method for their ability to detect three types of artificially added confounders in a chest x-ray diagnosis task. We find that the post-hoc technique SHAP, as well as the inherently interpretable Attri-Net provide the best performance and can be used to reliably identify faulty model behavior.

翻译：尽管深度神经网络模型在分类任务中表现出无与伦比的性能，但它们容易学习数据中的虚假相关性。如果测试数据与训练数据分布相同，这种对混杂信息的依赖可能难以通过性能指标检测出来。可解释机器学习方法（如事后解释方法或本质可解释分类器）有望识别模型推理中的错误。然而，关于这些技术是否真正具备此项能力，现有证据尚不明确。本文提出了一种严格的评估策略，用以衡量解释技术正确识别虚假相关性的能力。基于该策略，我们评估了五种事后解释技术与一种本质可解释方法在胸部X光诊断任务中检测三种人为添加混杂因素的能力。结果表明，事后解释技术SHAP以及本质可解释方法Attri-Net表现最佳，可用于可靠地识别错误模型行为。