CSCW studies have increasingly explored AI's role in enhancing communication efficiency and productivity in collaborative tasks. AI tools such as chatbots, smart replies, and language models aim to optimize conversation management and improve team performance. Early AI assistants, such as Gmail smart reply, were limited by predefined knowledge bases and decision trees. However, the advent of large language models (LLMs) such as ChatGPT has revolutionized AI assistants, employing advanced deep learning architecture to generate context-aware, coherent, and personalized responses. Consequently, ChatGPT-based AI assistants provide a more natural and efficient user experience across various tasks and domains. In this paper, we formalize the concept of AI Collaborative Tools (ACT) as AI technologies in human collaborative work and discuss how the emergence of ChatGPT has transformed the AI landscape and increased focus on ACT for improving team performance. Meanwhile, we present an LLM-based Smart Reply (LSR) system utilizing the ChatGPT API to generate personalized responses in daily collaborative scenarios, considering context, tone, and communication style. Our two-step process involves generating a preliminary response type (e.g., Agree, Disagree) to provide a generalized direction for message generation, thus reducing response drafting time. We conducted an experiment in which participants completed simulated work tasks, involving Google Calendar manipulation and a double-back N-back test, while interacting with researchers posing as teammates requesting scheduling changes. Our findings indicate that the AI teammate increases perceived performance and reduces mental demand, as measured by the NASA TLX, and improves performance in the N-back task. We also provide qualitative feedback on participants' experiences working with the AI teammate.
翻译:CSCW研究日益探索人工智能在提升协作任务中沟通效率与生产力方面的作用。诸如聊天机器人、智能回复和语言模型等AI工具,旨在优化对话管理并提升团队绩效。早期的AI助手,如Gmail智能回复,受限于预定义的知识库和决策树。然而,以ChatGPT为代表的大语言模型(LLMs)的出现,通过采用先进的深度学习架构生成上下文感知、连贯且个性化的回复,彻底改变了AI助手的格局。因此,基于ChatGPT的AI助手在各种任务和领域提供了更自然、更高效的用户体验。本文中,我们将AI协作工具(ACT)概念化为人类协作工作中的AI技术,并探讨ChatGPT的出现如何改变AI格局,并增强了对ACT用于提升团队绩效的关注。同时,我们提出了一种基于LLM的智能回复(LSR)系统,该系统利用ChatGPT API在日常协作场景中生成考虑上下文、语气和沟通风格的个性化回复。我们的两步骤流程包括首先生成一个初步回复类型(例如同意、不同意),为消息生成提供总体方向,从而减少回复起草时间。我们开展了一项实验,参与者需完成模拟工作任务(包括Google日历操作和双回溯N-back测试),同时与扮演请求日程变更的队友的研究人员互动。研究结果表明,AI队友提升了感知绩效并降低了心理负荷(如NASA TLX所测),并在N-back任务中改善了表现。我们还提供了关于参与者与AI队友协作体验的定性反馈。