Explainable artificial intelligence techniques are evolving at breakneck speed, but suitable evaluation approaches currently lag behind. With explainers becoming increasingly complex and a lack of consensus on how to assess their utility, it is challenging to judge the benefit and effectiveness of different explanations. To address this gap, we take a step back from complex predictive algorithms and instead look into explainability of simple mathematical models. In this setting, we aim to assess how people perceive comprehensibility of different model representations such as mathematical formulation, graphical representation and textual summarisation (of varying scope). This allows diverse stakeholders -- engineers, researchers, consumers, regulators and the like -- to judge intelligibility of fundamental concepts that more complex artificial intelligence explanations are built from. This position paper charts our approach to establishing appropriate evaluation methodology as well as a conceptual and practical framework to facilitate setting up and executing relevant user studies.

翻译：可解释人工智能技术正在飞速发展，但合适的评估方法目前却滞后。随着解释器日益复杂且缺乏评估其效用的共识，判断不同解释的益处和有效性颇具挑战性。为填补这一空白，我们从复杂的预测算法退一步，转而探究简单数学模型的可解释性。在此背景下，我们旨在评估人们如何感知不同模型表示的可理解性，例如数学公式、图形化表示和（不同范围的）文本摘要。这使得多元利益相关者——工程师、研究人员、消费者、监管者等——能够判断构成更复杂人工智能解释的基础概念的清晰度。本文作为立场论文，概述了我们建立适当评估方法以及概念和实践框架的思路，以促进相关用户研究的设定和执行。