Despite the increasing relevance of explainable AI, assessing the quality of explanations remains a challenging issue. Due to the high costs associated with human-subject experiments, various proxy metrics are often used to approximately quantify explanation quality. Generally, one possible interpretation of the quality of an explanation is its inherent value for teaching a related concept to a student. In this work, we extend artificial simulatability studies to the domain of graph neural networks. Instead of costly human trials, we use explanation-supervisable graph neural networks to perform simulatability studies to quantify the inherent usefulness of attributional graph explanations. We perform an extensive ablation study to investigate the conditions under which the proposed analyses are most meaningful. We additionally validate our methods applicability on real-world graph classification and regression datasets. We find that relevant explanations can significantly boost the sample efficiency of graph neural networks and analyze the robustness towards noise and bias in the explanations. We believe that the notion of usefulness obtained from our proposed simulatability analysis provides a dimension of explanation quality that is largely orthogonal to the common practice of faithfulness and has great potential to expand the toolbox of explanation quality assessments, specifically for graph explanations.

翻译：尽管可解释人工智能的重要性日益增加，评估解释质量仍是一项具有挑战性的任务。由于人类受试者实验成本高昂，常采用各种代理指标来近似量化解释质量。通常，解释质量的一种可能解释是其对于向学生传授相关概念的固有能力。在本研究中，我们将人工可模拟性研究扩展到图神经网络领域。通过使用可解释监督的图神经网络替代昂贵的人类试验，执行可模拟性研究以量化图归因解释的内在有用性。我们进行了广泛的消融研究，以探究提出分析方法最具意义的条件，并在真实世界的图分类与回归数据集上验证了方法的适用性。研究发现，高质量解释能显著提升图神经网络的样本效率，并分析了其对解释中噪声与偏差的鲁棒性。我们认为，通过所提出的可模拟性分析获得的有用性概念，为解释质量提供了一个与传统忠实度实践基本正交的新维度，并具有极大潜力扩展解释质量评估工具库，尤其是在图解释领域。