Collaborative Qualitative Analysis (CQA) can enhance qualitative analysis rigor and depth by incorporating varied viewpoints. Nevertheless, ensuring a rigorous CQA procedure itself can be both demanding and costly. To lower this bar, we take a theoretical perspective to design the CollabCoder workflow, that integrates Large Language Models (LLMs) into key inductive CQA stages: independent open coding, iterative discussions, and final codebook creation. In the open coding phase, CollabCoder offers AI-generated code suggestions and records decision-making data. During discussions, it promotes mutual understanding by sharing this data within the coding team and using quantitative metrics to identify coding (dis)agreements, aiding in consensus-building. In the code grouping stage, CollabCoder provides primary code group suggestions, lightening the cognitive load of finalizing the codebook. A 16-user evaluation confirmed the effectiveness of CollabCoder, demonstrating its advantages over existing software and providing empirical insights into the role of LLMs in the CQA practice.
翻译:协作定性分析(CQA)可通过融入多元视角来增强定性分析的严谨性与深度。然而,确保CQA流程本身的严谨性既要求高专业素养又代价不菲。为降低这一门槛,我们从理论视角设计了CollabCoder工作流,将大语言模型(LLMs)整合至归纳式CQA的关键阶段:独立开放式编码、迭代讨论及最终编码本生成。在开放式编码阶段,CollabCoder提供AI生成的代码建议并记录决策过程数据。讨论环节中,通过将决策数据在编码团队内共享并借助量化指标识别编码(不)一致性,该工具促进成员间相互理解,助力达成共识。在代码分组阶段,CollabCoder提供初步代码分组建议,减轻最终确定编码本的认知负荷。一项16人用户评估验证了CollabCoder的有效性,展示了其相对于现有软件的优势,并为LLMs在CQA实践中的作用提供了实证洞见。