The field of 'explainable' artificial intelligence (XAI) has produced highly cited methods that seek to make the decisions of complex machine learning (ML) methods 'understandable' to humans, for example by attributing 'importance' scores to input features. Yet, a lack of formal underpinning leaves it unclear as to what conclusions can safely be drawn from the results of a given XAI method and has also so far hindered the theoretical verification and empirical validation of XAI methods. This means that challenging non-linear problems, typically solved by deep neural networks, presently lack appropriate remedies. Here, we craft benchmark datasets for three different non-linear classification scenarios, in which the important class-conditional features are known by design, serving as ground truth explanations. Using novel quantitative metrics, we benchmark the explanation performance of a wide set of XAI methods across three deep learning model architectures. We show that popular XAI methods are often unable to significantly outperform random performance baselines and edge detection methods. Moreover, we demonstrate that explanations derived from different model architectures can be vastly different; thus, prone to misinterpretation even under controlled conditions.

翻译：“可解释”人工智能领域已产生高引用量的方法，旨在通过为输入特征赋予“重要性”分数等方式，使复杂机器学习方法的决策对人类“可理解”。然而，由于缺乏形式化基础，目前尚不明确从特定XAI方法的结果中能安全得出何种结论，这亦阻碍了XAI方法的理论验证与实证检验。这意味着，通常由深度神经网络解决的具有挑战性的非线性问题目前缺乏适当的补救措施。本文针对三种不同的非线性分类场景构建基准数据集，其中重要的类条件特征通过设计已知，可作为解释的真值。利用新颖的定量指标，我们对涵盖三种深度学习模型架构的广泛XAI方法的解释性能进行基准测试。结果表明，流行的XAI方法通常无法显著超越随机性能基线及边缘检测方法。此外，我们证明，从不同模型架构得出的解释可能存在巨大差异，因此即使在受控条件下也易于产生误解。