The use of large language models (LLMs) for qualitative analysis is gaining attention in various fields, including software engineering, where qualitative methods are essential for understanding human and social factors. This study aimed to investigate how LLMs are currently used in qualitative analysis and their potential applications in software engineering research, focusing on the benefits, limitations, and practices associated with their use. A systematic mapping study was conducted, analyzing 21 relevant studies to explore reported uses of LLMs for qualitative analysis. The findings indicate that LLMs are primarily used for tasks such as coding, thematic analysis, and data categorization, offering benefits like increased efficiency and support for new researchers. However, limitations such as output variability, challenges in capturing nuanced perspectives, and ethical concerns related to privacy and transparency were also identified. The study emphasizes the need for structured strategies and guidelines to optimize LLM use in qualitative research within software engineering, enhancing their effectiveness while addressing ethical considerations. While LLMs show promise in supporting qualitative analysis, human expertise remains crucial for interpreting data, and ongoing exploration of best practices will be vital for their successful integration into empirical software engineering research.
翻译:大型语言模型(LLMs)在定性分析中的应用正受到包括软件工程在内的多个领域的关注,其中定性方法对于理解人因与社会因素至关重要。本研究旨在调查LLMs目前在定性分析中的使用情况及其在软件工程研究中的潜在应用,重点关注其使用的优势、局限与实践。通过开展系统性图谱研究,分析了21项相关研究以探究LLMs在定性分析中的已报告用途。研究结果表明,LLMs主要用于编码、主题分析和数据分类等任务,其优势包括提升效率及支持新研究人员。然而,研究也识别出若干局限,如输出结果的可变性、捕捉细微视角的挑战,以及与隐私和透明度相关的伦理问题。本研究强调需要制定结构化策略与指导原则,以优化LLMs在软件工程定性研究中的应用,在提升其效能的同时应对伦理考量。尽管LLMs在支持定性分析方面展现出潜力,但人类专业知识在数据解读中仍至关重要,持续探索最佳实践对于其成功融入实证软件工程研究具有重要意义。