As human-AI cooperation becomes increasingly prevalent, reliable instruments for assessing the subjective quality of cooperative human-AI interaction are needed. We introduce two theoretically grounded scales: the Perceived Cooperativity Scale (PCS), grounded in joint activity theory, and the Teaming Perception Scale (TPS), grounded in evolutionary cooperation theory. The PCS captures an agent's perceived cooperative capability and practice within a single interaction sequence; the TPS captures the emergent sense of teaming arising from mutual contribution and support. Both scales were adapted for human-human cooperation to enable cross-agent comparisons. Across three studies (N = 409) encompassing a cooperative card game, LLM interaction, and a decision-support system, analyses of dimensionality, reliability, and validity indicated that both scales successfully differentiated between cooperation partners of varying cooperative quality and showed construct validity in line with expectations. The scales provide a basis for empirical investigation and system evaluation across a wide range of human-AI cooperation contexts.
翻译:随着人机协作日益普遍,我们需要可靠的工具来评估人机协作互动的主观质量。我们提出了两个基于理论的量表:基于联合行动理论的感知协作性量表(PCS),以及基于演化合作理论的团队感量表(TPS)。PCS衡量的是在单个交互序列中,智能体被感知到的协作能力与实践;TPS衡量的是由相互贡献和支持产生的涌现性团队感。两个量表均改编自人际协作版本,以便进行跨智能体比较。在三项研究(N=409)中,涵盖合作纸牌游戏、大语言模型交互和决策支持系统,对维度、信度和效度的分析表明,两个量表能够有效区分不同协作质量的合作伙伴,并展现出符合预期的建构效度。这些量表为广泛的人机协作情境下的实证研究和系统评估提供了基础。