Qualitative analysis is critical to understanding human datasets in many social science disciplines. A central method in this process is inductive coding, where researchers identify and interpret codes directly from the datasets themselves. Yet, this exploratory approach poses challenges for meeting methodological expectations (such as ``depth'' and ``variation''), especially as researchers increasingly adopt Generative AI (GAI) for support. Ground-truth-based metrics are insufficient because they contradict the exploratory nature of inductive coding, while manual evaluation can be labor-intensive. This paper presents a theory-informed computational method for measuring inductive coding results from humans and GAI. Our method first merges individual codebooks using an LLM-enriched algorithm. It measures each coder's contribution against the merged result using four novel metrics: Coverage, Overlap, Novelty, and Divergence. Through two experiments on a human-coded online conversation dataset, we 1) reveal the merging algorithm's impact on metrics; 2) validate the metrics' stability and robustness across multiple runs and different LLMs; and 3) showcase the metrics' ability to diagnose coding issues, such as excessive or irrelevant (hallucinated) codes. Our work provides a reliable pathway for ensuring methodological rigor in human-AI qualitative analysis.

翻译：定性分析对于理解许多社会科学学科中的人类数据集至关重要。这一过程的核心方法是归纳编码，即研究者直接从数据集中识别并解读编码。然而，这种探索性方法在满足方法论期望（如“深度”和“变异性”）方面存在挑战，尤其是在研究者越来越多地采用生成式人工智能（GAI）辅助的情况下。基于真实基准的指标并不充分，因为它们与归纳编码的探索性本质相悖，而人工评估则可能劳动强度大。本文提出了一种受理论启发的计算测量方法，用于评估人类与GAI的归纳编码结果。我们的方法首先使用大语言模型增强的算法合并个体编码本。然后，通过四种新颖指标（覆盖率、重叠度、新颖性和发散性）测量每个编码者相对于合并结果的贡献。通过对一个由人工标注的在线对话数据集进行两项实验，我们：1）揭示了合并算法对指标的影响；2）验证了指标在多次运行和不同大语言模型下的稳定性与鲁棒性；3）展示了指标诊断编码问题（如编码过多或不相关（幻觉））的能力。我们的工作为确保人机协作定性分析的方法论严谨性提供了可靠路径。