Obtaining stakeholders' diverse experiences and opinions about current policy in a timely manner is crucial for policymakers to identify strengths and gaps in resource allocation, thereby supporting effective policy design and implementation. However, manually coding even moderately sized interview texts or open-ended survey responses from stakeholders can often be labor-intensive and time-consuming. This study explores the integration of Large Language Models (LLMs)--like GPT-4--with human expertise to enhance text analysis of stakeholder interviews regarding K-12 education policy within one U.S. state. Employing a mixed-methods approach, human experts developed a codebook and coding processes as informed by domain knowledge and unsupervised topic modeling results. They then designed prompts to guide GPT-4 analysis and iteratively evaluate different prompts' performances. This combined human-computer method enabled nuanced thematic and sentiment analysis. Results reveal that while GPT-4 thematic coding aligned with human coding by 77.89% at specific themes, expanding to broader themes increased congruence to 96.02%, surpassing traditional Natural Language Processing (NLP) methods by over 25%. Additionally, GPT-4 is more closely matched to expert sentiment analysis than lexicon-based methods. Findings from quantitative measures and qualitative reviews underscore the complementary roles of human domain expertise and automated analysis as LLMs offer new perspectives and coding consistency. The human-computer interactive approach enhances efficiency, validity, and interpretability of educational policy research.

翻译：及时获取利益相关者关于现行政策的多元经验与观点，对于政策制定者识别资源配置的优势与缺口、从而支撑有效的政策设计与实施至关重要。然而，即使是对利益相关者中等规模访谈文本或开放式调查问卷的回复进行人工编码，也往往费时费力。本研究探索将大语言模型（LLMs，如GPT-4）与人类专业知识相结合，以增强对美国某州K-12教育政策利益相关者访谈的文本分析。采用混合方法，人类专家基于领域知识和无监督主题建模结果开发了编码手册与编码流程，进而设计提示词以引导GPT-4分析，并迭代评估不同提示词的表现。这种人机结合的方法实现了细致的主题与情感分析。结果表明，GPT-4在特定主题上的主题编码与人工编码一致性达77.89%，扩展到更宽泛主题时一致性增至96.02%，超越传统自然语言处理（NLP）方法超25%。此外，GPT-4的情感分析与专家情感分析的匹配度优于基于词典的方法。量化指标与定性评估的发现均强调，人类领域专长与自动化分析具有互补作用——大语言模型提供了新视角与编码一致性。人机交互方法提升了教育政策研究的效率、效度与可解释性。