The field of explainable artificial intelligence (XAI) aims to uncover the inner workings of complex deep neural models. While being crucial for safety-critical domains, XAI inherently lacks ground-truth explanations, making its automatic evaluation an unsolved problem. We address this challenge by proposing a novel synthetic vision dataset, named FunnyBirds, and accompanying automatic evaluation protocols. Our dataset allows performing semantically meaningful image interventions, e.g., removing individual object parts, which has three important implications. First, it enables analyzing explanations on a part level, which is closer to human comprehension than existing methods that evaluate on a pixel level. Second, by comparing the model output for inputs with removed parts, we can estimate ground-truth part importances that should be reflected in the explanations. Third, by mapping individual explanations into a common space of part importances, we can analyze a variety of different explanation types in a single common framework. Using our tools, we report results for 24 different combinations of neural models and XAI methods, demonstrating the strengths and weaknesses of the assessed methods in a fully automatic and systematic manner.

翻译：可解释人工智能（XAI）领域旨在揭示复杂深度神经模型的内部工作原理。尽管XAI对安全关键领域至关重要，但其本质上缺乏真实标注的解释，因此自动评估仍是一个未解决的问题。我们通过提出一个名为FunnyBirds的新型合成视觉数据集及配套的自动评估协议来应对这一挑战。我们的数据集允许执行语义上有意义的图像干预操作，例如移除单个物体部件，这具有三个重要影响。首先，它能够在部件级别分析解释，这比现有在像素级别评估的方法更接近人类理解。其次，通过比较移除部件前后的模型输出，我们可以估算出应在解释中反映的部件级真实重要性。第三，通过将单个解释映射到部件重要性的共同空间，我们可以在单一统一框架内分析多种不同类型的解释。利用我们的工具，我们报告了24种不同神经模型与XAI方法组合的结果，以完全自动化且系统化的方式展示了被评估方法的优势与不足。