Explanations of AI systems rarely address the information needs of people affected by algorithmic decision-making (ADM). This gap between conveyed information and information that matters to affected stakeholders can impede understanding and adherence to regulatory frameworks such as the AI Act. To address this gap, we present the "XAI Novice Question Bank": A catalog of affected stakeholders' information needs in two ADM use cases (employment prediction and health monitoring), covering the categories data, system context, system usage, and system specifications. Information needs were gathered in an interview study where participants received explanations in response to their inquiries. Participants further reported their understanding and decision confidence, showing that while confidence tended to increase after receiving explanations, participants also met understanding challenges, such as being unable to tell why their understanding felt incomplete. Explanations further influenced participants' perceptions of the systems' risks and benefits, which they confirmed or changed depending on the use case. When risks were perceived as high, participants expressed particular interest in explanations about intention, such as why and to what end a system was put in place. With this work, we aim to support the inclusion of affected stakeholders into explainability by contributing an overview of information and challenges relevant to them when deciding on the adoption of ADM systems. We close by summarizing our findings in a list of six key implications that inform the design of future explanations for affected stakeholder audiences.
翻译:人工智能系统的解释很少满足受算法决策影响人群的信息需求。这种传达信息与受影响利益相关者关心信息之间的差距,可能阻碍对AI法案等监管框架的理解和遵守。为弥补这一空白,我们提出了“XAI新手问题银行”:一个涵盖两种算法决策用例(就业预测和健康监测)中受影响利益相关者信息需求的目录,涉及数据、系统上下文、系统使用和系统规范等类别。信息需求通过访谈研究收集,参与者在提问后获得解释。参与者还报告了他们的理解和决策信心,结果显示虽然收到解释后信心往往增加,但参与者仍面临理解挑战,例如无法说清为何感觉理解不完整。解释进一步影响了参与者对系统风险和收益的认知,他们根据用例确认或改变了这些认知。当风险被视为高时,参与者对关于意图的解释特别感兴趣,例如系统为何以及为何目的而部署。通过这项工作,我们旨在通过概述受影响利益相关者在决定是否采用算法决策系统时相关的信息和挑战,支持将这些人群纳入可解释性之中。最后,我们根据研究结果总结出六项关键启示,为未来面向受影响利益相关者受众的解释设计提供指导。