Large Language Models (LLMs) have significantly impacted numerous domains, including Software Engineering (SE). Many recent publications have explored LLMs applied to various SE tasks. Nevertheless, a comprehensive understanding of the application, effects, and possible limitations of LLMs on SE is still in its early stages. To bridge this gap, we conducted a systematic literature review (SLR) on LLM4SE, with a particular focus on understanding how LLMs can be exploited to optimize processes and outcomes. We select and analyze 395 research papers from January 2017 to January 2024 to answer four key research questions (RQs). In RQ1, we categorize different LLMs that have been employed in SE tasks, characterizing their distinctive features and uses. In RQ2, we analyze the methods used in data collection, preprocessing, and application, highlighting the role of well-curated datasets for successful LLM for SE implementation. RQ3 investigates the strategies employed to optimize and evaluate the performance of LLMs in SE. Finally, RQ4 examines the specific SE tasks where LLMs have shown success to date, illustrating their practical contributions to the field. From the answers to these RQs, we discuss the current state-of-the-art and trends, identifying gaps in existing research, and flagging promising areas for future study. Our artifacts are publicly available at https://github.com/xinyi-hou/LLM4SE_SLR.
翻译:大型语言模型(LLM)已对包括软件工程(SE)在内的众多领域产生显著影响。近年来众多研究探讨了LLM在各类软件工程任务中的应用,然而,关于LLM在软件工程领域中的应用效果及潜在局限性的全面认知仍处于初期阶段。为填补这一空白,我们对LLM4SE领域开展了系统性文献综述(SLR),重点探究如何利用LLM优化流程与成果。我们筛选并分析了2017年1月至2024年1月间的395篇研究论文,以回答四个关键研究问题(RQ)。RQ1对应用于软件工程任务的不同LLM进行归类,阐释其独特特征与使用方式;RQ2分析数据收集、预处理与应用过程中的方法,强调优质数据集对LLM成功应用于软件工程的关键作用;RQ3探讨优化与评估LLM在软件工程中表现所采用的策略;RQ4最后考察当前LLM已取得成效的特定软件工程任务,阐明其对领域的实际贡献。基于这些研究问题的答案,我们讨论了当前研究现状与趋势,指出现有研究空白,并标注了未来研究的潜在方向。我们的相关资源已公开于https://github.com/xinyi-hou/LLM4SE_SLR。