Static analysis is a widely used technique in software engineering for identifying and mitigating bugs. However, a significant hurdle lies in achieving a delicate balance between precision and scalability. Large Language Models (LLMs) offer a promising alternative, as recent advances demonstrate remarkable capabilities in comprehending, generating, and even debugging code. Yet, the logic of bugs can be complex and require sophisticated reasoning and a large analysis scope spanning multiple functions. Therefore, at this point, LLMs are better used in an assistive role to complement static analysis. In this paper, we take a deep dive into the open space of LLM-assisted static analysis, using use-before-initialization (UBI) bugs as a case study. To this end, we develop LLift, a fully automated framework that interfaces with both a static analysis tool and an LLM. By carefully designing the framework and the prompts, we are able to overcome a number of challenges, including bug-specific modeling, the large problem scope, the non-deterministic nature of LLMs, etc. Tested in a real-world scenario analyzing nearly a thousand potential UBI bugs produced by static analysis, LLift demonstrates a potent capability, showcasing a reasonable precision (50%) and appearing to have no missing bugs. It even identified 13 previously unknown UBI bugs in the Linux kernel. This research paves the way for new opportunities and methodologies in using LLMs for bug discovery in extensive, real-world datasets.
翻译:静态分析是软件工程中广泛用于识别和缓解缺陷的技术。然而,在精度与可扩展性之间取得微妙平衡仍是一大难题。大语言模型(LLMs)提供了有前景的替代方案,近期研究进展表明其在代码理解、生成乃至调试方面展现出卓越能力。但缺陷逻辑可能复杂,需要跨多个函数的深度推理与大规模分析范围。因此,现阶段LLMs更适合作为静态分析的辅助工具。本文以未初始化使用(UBI)缺陷为案例,深入探索LLM辅助静态分析这一开放领域。为此,我们开发了LLift——一种完全自动化的框架,可同时对接静态分析工具与LLM。通过精心设计框架与提示词,我们克服了包括缺陷特异性建模、大规模问题空间、LLM非确定性等挑战。在真实场景中对静态分析产生的近千个潜在UBI缺陷进行测试,LLift展现出强大能力:达到合理精度(50%)且似乎无漏检,甚至在Linux内核中发现了13个此前未知的UBI缺陷。本研究表明,将LLMs应用于大规模真实数据集的缺陷发现具有全新机遇与方法论价值。