Despite the proliferation of explainable AI (XAI) methods, little is understood about end-users' explainability needs and behaviors around XAI explanations. To address this gap and contribute to understanding how explainability can support human-AI interaction, we conducted a mixed-methods study with 20 end-users of a real-world AI application, the Merlin bird identification app, and inquired about their XAI needs, uses, and perceptions. We found that participants desire practically useful information that can improve their collaboration with the AI, more so than technical system details. Relatedly, participants intended to use XAI explanations for various purposes beyond understanding the AI's outputs: calibrating trust, improving their task skills, changing their behavior to supply better inputs to the AI, and giving constructive feedback to developers. Finally, among existing XAI approaches, participants preferred part-based explanations that resemble human reasoning and explanations. We discuss the implications of our findings and provide recommendations for future XAI design.
翻译:尽管可解释人工智能(XAI)方法日益增多,但人们对于终端用户在XAI解释方面的可解释性需求及行为仍知之甚少。为填补这一空白并探讨可解释性如何支持人机交互,我们针对一款真实世界AI应用——Merlin鸟类识别应用的20名终端用户开展了混合方法研究,调查了他们对XAI的需求、使用情况及认知。研究发现,参与者更渴望获得能提升与AI协作效率的实用信息,而非系统技术细节。相应地,参与者计划将XAI解释用于除理解AI输出之外的多种目的:校准信任、提升任务技能、调整行为以向AI提供更优输入,以及向开发者提供建设性反馈。此外,在现有XAI方法中,参与者更偏好那些类似人类推理与解释的基于部分的解释方法。我们讨论了研究发现的启示,并为未来XAI设计提出了建议。