As public sector agencies rapidly introduce new AI tools in high-stakes domains like social services, it becomes critical to understand how decisions to adopt these tools are made in practice. We borrow from the anthropological practice to ``study up'' those in positions of power, and reorient our study of public sector AI around those who have the power and responsibility to make decisions about the role that AI tools will play in their agency. Through semi-structured interviews and design activities with 16 agency decision-makers, we examine how decisions about AI design and adoption are influenced by their interactions with and assumptions about other actors within these agencies (e.g., frontline workers and agency leaders), as well as those above (legal systems and contracted companies), and below (impacted communities). By centering these networks of power relations, our findings shed light on how infrastructural, legal, and social factors create barriers and disincentives to the involvement of a broader range of stakeholders in decisions about AI design and adoption. Agency decision-makers desired more practical support for stakeholder involvement around public sector AI to help overcome the knowledge and power differentials they perceived between them and other stakeholders (e.g., frontline workers and impacted community members). Building on these findings, we discuss implications for future research and policy around actualizing participatory AI approaches in public sector contexts.
翻译:随着公共部门机构在社会保障等高风险领域快速引入新型AI工具,理解这些工具在实际决策过程中如何被采纳变得至关重要。我们借鉴人类学中“向上研究”权力行使者的方法论,将公共部门AI研究视角转向那些有权且负责决定AI工具在机构中角色的决策者。通过对16位机构决策者进行半结构化访谈和设计活动,我们考察了AI设计与采纳决策如何受到他们与机构内部行动者(如一线工作者和机构领导)、上级系统(法律体系与承包公司)及受影响群体之间的互动和预设影响。通过聚焦这些权力关系网络,本研究发现基础设施、法律和社会因素如何构成障碍和抑制因素,阻碍更广泛的利益相关者参与AI设计与采纳决策。机构决策者期望获得更多关于公共部门AI利益相关者参与的实践支持,以克服他们与其他利益相关者(如一线工作者和受影响社区成员)之间存在的知识和权力差异。基于这些发现,我们讨论了在公共部门背景下实现参与式AI方法的未来研究与政策启示。