Explainable Artificial Intelligence (XAI) aims to improve the transparency of autonomous decision-making through explanations. Recent literature has emphasised users' need for holistic "multi-shot" explanations and personalised engagement with XAI systems. We refer to this user-centred interaction as an XAI Experience. Despite advances in creating XAI experiences, evaluating them in a user-centred manner has remained challenging. In response, we developed the XAI Experience Quality (XEQ) Scale. XEQ quantifies the quality of experiences across four dimensions: learning, utility, fulfilment and engagement. These contributions extend the state-of-the-art of XAI evaluation, moving beyond the one-dimensional metrics frequently developed to assess single-shot explanations. This paper presents the XEQ scale development and validation process, including content validation with XAI experts, and discriminant and construct validation through a large-scale pilot study. Our pilot study results offer strong evidence that establishes the XEQ Scale as a comprehensive framework for evaluating user-centred XAI experiences.

翻译：可解释人工智能（XAI）旨在通过解释提升自主决策的透明度。近期文献强调用户对整体性“多轮次”解释以及与XAI系统个性化交互的需求。我们将这种以用户为中心的交互称为XAI体验。尽管在构建XAI体验方面已取得进展，但以用户为中心的方式评估这些体验仍面临挑战。为此，我们开发了XAI体验质量（XEQ）量表。XEQ从四个维度量化体验质量：学习性、实用性、满足感与参与度。这些贡献拓展了XAI评估的前沿方法，超越了当前常被用于评估单轮次解释的一维指标。本文阐述了XEQ量表的开发与验证过程，包括与XAI专家的内容效度验证，以及通过大规模试点研究进行的区分效度与结构效度验证。我们的试点研究结果提供了有力证据，证实XEQ量表可作为评估以用户为中心的XAI体验的综合性框架。