In recent years, artificial intelligence has had a conspicuous growth in almost every aspect of life. One of the most applicable areas is security code review, in which a lot of AI-based tools and approaches have been proposed. Recently, ChatGPT has caught a huge amount of attention with its remarkable performance in following instructions and providing a detailed response. Regarding the similarities between natural language and code, in this paper, we study the feasibility of using ChatGPT for vulnerability detection in Python source code. Toward this goal, we feed an appropriate prompt along with vulnerable data to ChatGPT and compare its results on two datasets with the results of three widely used Static Application Security Testing tools (Bandit, Semgrep and SonarQube). We implement different kinds of experiments with ChatGPT and the results indicate that ChatGPT reduces the false positive and false negative rates and has the potential to be used for Python source code vulnerability detection.
翻译:近年来,人工智能在生活的几乎各个方面都取得了显著增长。其中一个最具应用前景的领域是安全代码审查,其中已提出了大量基于人工智能的工具和方法。近期,ChatGPT因其在遵循指令和提供详细回答方面的卓越性能而备受关注。鉴于自然语言与代码之间的相似性,本文研究了使用ChatGPT检测Python源代码漏洞的可行性。为此,我们向ChatGPT输入适当提示及漏洞数据,并在两个数据集上将其结果与三种广泛使用的静态应用安全测试工具(Bandit、Semgrep和SonarQube)的结果进行比较。我们使用ChatGPT开展了不同类型的实验,结果表明ChatGPT降低了假阳性率和假阴性率,具有用于Python源代码漏洞检测的潜力。