Research on Collaborative Problem Solving (CPS) has traditionally examined how humans rely on one another cognitively and socially to accomplish tasks together. With the rapid advancement of AI and large language models, however, a new question emerge: what happens to team dynamics when one of the "teammates" is not human? In this study, we investigate how the integration of an AI teammate -- a fully autonomous GPT-4 agent with social, cognitive, and affective capabilities -- shapes the socio-cognitive dynamics of CPS. We analyze discourse data collected from human-AI teaming (HAT) experiments conducted on a novel platform specifically designed for HAT research. Using two natural language processing (NLP) methods, specifically Linguistic Inquiry and Word Count (LIWC) and Group Communication Analysis (GCA), we found that AI teammates often assumed the role of dominant cognitive facilitators, guiding, planning, and driving group decision-making. However, they did so in a socially detached manner, frequently pushing agenda in a verbose and repetitive way. By contrast, humans working with AI used more language reflecting social processes, suggesting that they assumed more socially oriented roles. Our study highlights how learning analytics can provide critical insights into the socio-cognitive dynamics of human-AI collaboration.

翻译：协作问题解决（CPS）的传统研究主要探讨人类如何依赖彼此的认知与社会能力共同完成任务。然而，随着人工智能和大语言模型的快速发展，一个新的问题随之产生：当团队中的“成员”之一并非人类时，团队动态会发生何种变化？本研究探讨了集成一个AI队友——一个具备社交、认知与情感能力的全自主GPT-4智能体——如何塑造CPS的社会认知动态。我们分析了一个专门为人机协作（HAT）研究设计的新平台上开展HAT实验所收集的对话数据。通过运用两种自然语言处理（NLP）方法，即语言探索与词频统计（LIWC）和群体沟通分析（GCA），我们发现AI队友常常扮演主导性认知促进者的角色，引导、规划并推动群体决策。然而，它们以一种社会性疏离的方式行事，频繁地以冗长重复的方式推进议程。相比之下，与AI协作的人类使用了更多反映社会过程的语言，表明他们承担了更具社会导向性的角色。我们的研究凸显了学习分析如何能够为理解人机协作的社会认知动态提供关键见解。