While AI-assisted individual qualitative analysis has been substantially studied, AI-assisted collaborative qualitative analysis (CQA)-a process that involves multiple researchers working together to interpret data-remains relatively unexplored. After identifying CQA practices and design opportunities through formative interviews, we designed and implemented CoAIcoder, a tool leveraging AI to enhance human-to-human collaboration within CQA through four distinct collaboration methods. With a between-subject design, we evaluated CoAIcoder with 32 pairs of CQA-trained participants across common CQA phases under each collaboration method. Our findings suggest that while using a shared AI model as a mediator among coders could improve CQA efficiency and foster agreement more quickly in the early coding stage, it might affect the final code diversity. We also emphasize the need to consider the independence level when using AI to assist human-to-human collaboration in various CQA scenarios. Lastly, we suggest design implications for future AI-assisted CQA systems.
翻译:尽管AI辅助的个体定性分析已得到广泛研究,但AI辅助的协作式定性分析(CQA)——即多名研究者共同协作以解读数据的过程——仍相对未被探索。通过形成性访谈识别CQA实践与设计机会后,我们设计并实现了CoAIcoder,该工具通过四种不同的协作方法,利用AI增强CQA中的人类协作。采用被试间设计,我们评估了32组接受过CQA训练的参与者在各协作方法下常见CQA阶段的表现。研究结果表明,虽然将共享AI模型作为编码者之间的中介能提升CQA效率并在早期编码阶段加速达成共识,但可能影响最终编码的多样性。我们同时强调,在不同CQA场景下使用AI辅助人类协作时需考虑独立性水平。最后,我们为未来AI辅助的CQA系统提出了设计启示。