Large Language Models (LLMs) have emerged as powerful tools for automating various programming tasks, including security-related ones, such as detecting and fixing vulnerabilities. Despite their promising capabilities, when required to produce or modify pre-existing code, LLMs could introduce vulnerabilities unbeknown to the programmer. When analyzing code, they could miss clear vulnerabilities or signal nonexistent ones. In this Systematic Literature Review (SLR), we aim to investigate both the security benefits and potential drawbacks of using LLMs for a variety of code-related tasks. In particular, first we focus on the types of vulnerabilities that could be introduced by LLMs, when used for producing code. Second, we analyze the capabilities of LLMs to detect and fix vulnerabilities, in any given code, and how the prompting strategy of choice impacts their performance in these two tasks. Last, we provide an in-depth analysis on how data poisoning attacks on LLMs can impact performance in the aforementioned tasks.

翻译：大型语言模型（LLMs）已成为自动化各类编程任务的有力工具，包括与安全相关的任务，例如漏洞检测与修复。尽管其能力前景广阔，但在要求生成或修改已有代码时，LLMs可能会在程序员不知情的情况下引入漏洞。在分析代码时，它们可能遗漏明显的漏洞或误报不存在的漏洞。本系统性文献综述旨在探究将LLMs用于各类代码相关任务时，其带来的安全效益与潜在缺陷。具体而言，首先我们关注LLMs在生成代码时可能引入的漏洞类型。其次，我们分析LLMs在检测与修复任意给定代码中漏洞的能力，以及所选择的提示策略如何影响其在这两项任务中的表现。最后，我们深入分析了针对LLMs的数据投毒攻击如何影响其在前述任务中的性能。