The use of large language models (LLMs) for qualitative analysis is gaining attention in various fields, including software engineering, where qualitative methods are essential for understanding human and social factors. This study aimed to investigate how LLMs are currently used in qualitative analysis and their potential applications in software engineering research, focusing on the benefits, limitations, and practices associated with their use. A systematic mapping study was conducted, analyzing 21 relevant studies to explore reported uses of LLMs for qualitative analysis. The findings indicate that LLMs are primarily used for tasks such as coding, thematic analysis, and data categorization, offering benefits like increased efficiency and support for new researchers. However, limitations such as output variability, challenges in capturing nuanced perspectives, and ethical concerns related to privacy and transparency were also identified. The study emphasizes the need for structured strategies and guidelines to optimize LLM use in qualitative research within software engineering, enhancing their effectiveness while addressing ethical considerations. While LLMs show promise in supporting qualitative analysis, human expertise remains crucial for interpreting data, and ongoing exploration of best practices will be vital for their successful integration into empirical software engineering research.
翻译:大语言模型(LLMs)在定性分析中的应用正受到包括软件工程在内的多个领域的关注,其中定性方法对于理解人因与社会因素至关重要。本研究旨在调查LLMs目前在定性分析中的使用情况及其在软件工程研究中的潜在应用,重点关注其使用带来的益处、局限及相关实践。我们开展了一项系统性图谱研究,分析了21项相关研究,以探讨LLMs在定性分析中的已报告用途。研究结果表明,LLMs主要用于编码、主题分析和数据分类等任务,其益处包括提升效率及支持新研究人员。然而,研究也识别出一些局限性,例如输出结果的变异性、捕捉细微视角的挑战,以及与隐私和透明度相关的伦理问题。本研究强调,需要在软件工程领域的定性研究中制定结构化策略与指导方针,以优化LLMs的使用,在提升其有效性的同时应对伦理考量。尽管LLMs在支持定性分析方面展现出潜力,但人类专业知识在数据解读中仍至关重要,持续探索最佳实践对于LLMs成功融入实证软件工程研究将至关重要。