Semi-structured interviews (SSIs) are a commonly employed data-collection method in healthcare research, offering in-depth qualitative insights into subject experiences. Despite their value, the manual analysis of SSIs is notoriously time-consuming and labor-intensive, in part due to the difficulty of extracting and categorizing emotional responses, and challenges in scaling human evaluation for large populations. In this study, we develop RACER, a Large Language Model (LLM) based expert-guided automated pipeline that efficiently converts raw interview transcripts into insightful domain-relevant themes and sub-themes. We used RACER to analyze SSIs conducted with 93 healthcare professionals and trainees to assess the broad personal and professional mental health impacts of the COVID-19 crisis. RACER achieves moderately high agreement with two human evaluators (72%), which approaches the human inter-rater agreement (77%). Interestingly, LLMs and humans struggle with similar content involving nuanced emotional, ambivalent/dialectical, and psychological statements. Our study highlights the opportunities and challenges in using LLMs to improve research efficiency and opens new avenues for scalable analysis of SSIs in healthcare research.

翻译：半结构化访谈（SSIs）是医疗健康研究中常用的数据收集方法，能深入获取研究对象的主观体验定性洞察。然而，人工分析SSIs既耗时又费力，其难点部分源于情感反应的提取与分类困难，以及大规模人群评估的扩展性挑战。本研究开发了RACER——一种基于大语言模型（LLM）的专家引导自动化流水线，可高效地将原始访谈转录转化为富有洞察力的领域相关主题与子主题。我们运用RACER分析了93名医疗专业人员及实习生的SSIs数据，旨在评估COVID-19危机对个人及职业心理健康的广泛影响。RACER与两位人类评估者达到中等高度一致性（72%），接近人类评分者间的信度（77%）。值得注意的是，大语言模型与人类在涉及细微情感、矛盾/辩证关系及心理陈述的内容上存在相似的识别困难。本研究揭示了利用大语言模型提升研究效率的机遇与挑战，为医疗健康研究中SSIs的可扩展分析开辟了新路径。