We introduce SkipAnalyzer, a large language model (LLM)-powered tool for static code analysis. SkipAnalyzer has three components: 1) an LLM-based static bug detector that scans source code and reports specific types of bugs, 2) an LLM-based false-positive filter that can identify false-positive bugs in the results of static bug detectors (e.g., the result of step 1) to improve detection accuracy, and 3) an LLM-based patch generator that can generate patches for the detected bugs above. As a proof-of-concept, SkipAnalyzer is built on ChatGPT, which has exhibited outstanding performance in various software engineering tasks. To evaluate SkipAnalyzer, we focus on two types of typical and critical bugs that are targeted by static bug detection, i.e., Null Dereference and Resource Leak as subjects. We employ Infer to aid the gathering of these two bug types from 10 open-source projects. Consequently, our experiment dataset contains 222 instances of Null Dereference bugs and 46 instances of Resource Leak bugs. Our study demonstrates that SkipAnalyzer achieves remarkable performance in the mentioned static analysis tasks, including bug detection, false-positive warning removal, and bug repair. In static bug detection, SkipAnalyzer achieves accuracy values of up to 68.37% for detecting Null Dereference bugs and 76.95% for detecting Resource Leak bugs, improving the precision of the current leading bug detector, Infer, by 12.86% and 43.13%, respectively. For removing false-positive warnings, SkipAnalyzer can reach a precision of up to 93.88% for Null Dereference bugs and 63.33% for Resource Leak bugs. Additionally, SkipAnalyzer surpasses state-of-the-art false-positive warning removal tools. Furthermore, in bug repair, SkipAnalyzer can generate syntactically correct patches to fix its detected bugs with a success rate of up to 97.30%.
翻译:我们介绍SkipAnalyzer,一款基于大语言模型(LLM)的静态代码分析工具。SkipAnalyzer包含三个组件:1)基于LLM的静态缺陷检测器,可扫描源代码并报告特定类型的缺陷;2)基于LLM的误报过滤器,能够识别静态缺陷检测器结果(如步骤1的结果)中的误报缺陷,以提高检测精度;3)基于LLM的补丁生成器,可为上述检测到的缺陷生成修复补丁。作为概念验证,SkipAnalyzer基于ChatGPT构建,后者在各项软件工程任务中展现出卓越性能。为评估SkipAnalyzer,我们选取了静态缺陷检测所针对的两类典型且关键的缺陷——空指针解引用与资源泄漏作为研究对象。借助Infer工具从10个开源项目中收集这两类缺陷,最终实验数据集包含222个空指针解引用实例和46个资源泄漏实例。研究表明,SkipAnalyzer在所述静态分析任务(包括缺陷检测、误报警告消除及缺陷修复)中表现卓著。在静态缺陷检测中,SkipAnalyzer对空指针解引用缺陷的检测准确率高达68.37%,对资源泄漏缺陷的检测准确率达76.95%,分别将当前领先的缺陷检测器Infer的精度提升了12.86%和43.13%。在消除误报警告方面,SkipAnalyzer对空指针解引用缺陷的精度可达93.88%,对资源泄漏缺陷的精度达63.33%,性能超越现有最先进的误报警告消除工具。此外,在缺陷修复中,SkipAnalyzer能够为其检测到的缺陷生成语法正确的修复补丁,成功率高达97.30%。