Large Language Models (LLMs) have significantly impacted numerous domains, notably including Software Engineering (SE). Nevertheless, a well-rounded understanding of the application, effects, and possible limitations of LLMs within SE is still in its early stages. To bridge this gap, our systematic literature review takes a deep dive into the intersection of LLMs and SE, with a particular focus on understanding how LLMs can be exploited in SE to optimize processes and outcomes. Through a comprehensive review approach, we collect and analyze a total of 229 research papers from 2017 to 2023 to answer four key research questions (RQs). In RQ1, we categorize and provide a comparative analysis of different LLMs that have been employed in SE tasks, laying out their distinctive features and uses. For RQ2, we detail the methods involved in data collection, preprocessing, and application in this realm, shedding light on the critical role of robust, well-curated datasets for successful LLM implementation. RQ3 allows us to examine the specific SE tasks where LLMs have shown remarkable success, illuminating their practical contributions to the field. Finally, RQ4 investigates the strategies employed to optimize and evaluate the performance of LLMs in SE, as well as the common techniques related to prompt optimization. Armed with insights drawn from addressing the aforementioned RQs, we sketch a picture of the current state-of-the-art, pinpointing trends, identifying gaps in existing research, and flagging promising areas for future study.
翻译:大语言模型(LLMs)已显著影响众多领域,尤其是软件工程(SE)。然而,关于LLMs在SE中的应用、效果及潜在局限性的全面理解仍处于早期阶段。为填补这一空白,本系统性文献综述深入探讨LLMs与SE的交集,重点关注如何利用LLMs优化SE流程与成果。通过系统化综述方法,我们收集并分析了2017年至2023年间共计229篇研究论文,以回答四个关键研究问题(RQs)。在RQ1中,我们对SE任务中采用的各类LLMs进行分类与比较分析,阐明其独特特征与用途。针对RQ2,我们详述了该领域中数据收集、预处理及应用方法,揭示高质量、精心整理的数据集对成功实施LLMs的关键作用。RQ3使我们得以审视LLMs已展现显著成效的特定SE任务,阐明其对该领域的实际贡献。最后,RQ4探讨了优化与评估LLMs在SE中性能的策略,以及提示工程相关的常见技术。基于解答上述RQs所获得的洞见,我们描绘了当前技术发展现状,识别研究趋势与现存空白,并指出未来研究中有前景的方向。