Despite the continued anthropomorphization of AI systems, the potential impact of racialization during human-AI interaction is understudied. This study explores how human-AI cooperation may be impacted by the belief that data used to train an AI system is racialized, that is, it was trained on data from a specific group of people. During this study, participants completed a human-AI cooperation task using the Pig Chase game. Participants of different self-identified demographics interacted with AI agents whose perceived racial identities were manipulated, allowing us to assess how sociocultural perspectives influence the decision-making of participants in the game. After the game, participants completed a survey questionnaire to explain the strategies they used while playing the game and to understand the perceived intelligence of their AI teammates. Statistical analysis of task behavior data revealed a statistically significant effect of the participant's demographic, as well as the interaction between this self-identified demographic and the treatment condition (i.e., the perceived demographic of the agent). The results indicated that Non-White participants viewed AI agents racialized as White in a positive way compared to AI agents racialized as Black. Both Black and White participants viewed the AI agent in the control treatment in a negative way. A baseline cognitive model of the task using ACT-R cognitive architecture was used to understand a cognitive-level, process-based explanation of the participants' perspectives based on results found from the study. This model helps us better understand the factors affecting the decision-making strategies of the game participants. Results from analysis of these data, as well as cognitive modeling, indicate a need to expand understanding of the ways racialization (whether implicit or explicit) impacts interaction with AI systems.

翻译：尽管AI系统持续被赋予拟人化特征，但人机交互过程中种族化的潜在影响尚未得到充分研究。本研究探讨了当人类认为AI系统的训练数据具有种族化特征（即使用特定人群数据进行训练）时，将如何影响人类与AI的合作关系。本研究要求参与者通过Pig Chase游戏完成人机协作任务。不同自我认同人口特征的参与者与感知种族身份受操控的AI智能体进行交互，使我们能够评估社会文化视角如何影响参与者在游戏中的决策行为。游戏结束后，参与者通过问卷调查解释其游戏策略，并评估其对AI队友智能水平的感知。对任务行为数据的统计分析显示，参与者的人口特征及其与实验条件（即智能体的感知人口特征）的交互作用均产生统计学显著影响。结果表明：与非黑人AI智能体相比，非白人参与者对种族化为白人的AI智能体持有更积极的态度；黑人与白人参与者均对对照组AI智能体持消极态度。本研究基于ACT-R认知架构构建了任务的基础认知模型，结合研究结果从认知层面、基于过程的视角解释参与者的心理机制。该模型有助于深入理解影响游戏参与者决策策略的因素。数据分析与认知建模结果表明，亟需拓展对种族化（无论隐性或显性）影响AI系统交互机制的理解。