Software vulnerabilities can cause numerous problems, including crashes, data loss, and security breaches. These issues greatly compromise quality and can negatively impact the market adoption of software applications and systems. Traditional bug-fixing methods, such as static analysis, often produce false positives. While bounded model checking, a form of Formal Verification (FV), can provide more accurate outcomes compared to static analyzers, it demands substantial resources and significantly hinders developer productivity. Can Machine Learning (ML) achieve accuracy comparable to FV methods and be used in popular instant code completion frameworks in near real-time? In this paper, we introduce SecureFalcon, an innovative model architecture with only 121 million parameters derived from the Falcon-40B model and explicitly tailored for classifying software vulnerabilities. To achieve the best performance, we trained our model using two datasets, namely the FormAI dataset and the FalconVulnDB. The FalconVulnDB is a combination of recent public datasets, namely the SySeVR framework, Draper VDISC, Bigvul, Diversevul, SARD Juliet, and ReVeal datasets. These datasets contain the top 25 most dangerous software weaknesses, such as CWE-119, CWE-120, CWE-476, CWE-122, CWE-190, CWE-121, CWE-78, CWE-787, CWE-20, and CWE-762. SecureFalcon achieves 94% accuracy in binary classification and up to 92% in multiclassification, with instant CPU inference times. It outperforms existing models such as BERT, RoBERTa, CodeBERT, and traditional ML algorithms, promising to push the boundaries of software vulnerability detection and instant code completion frameworks.

翻译：软件漏洞可能导致诸多问题，包括系统崩溃、数据丢失和安全漏洞。这些问题严重损害软件质量，并可能对软件应用和系统的市场采用产生负面影响。传统的缺陷修复方法（如静态分析）常产生误报。而有界模型检查作为形式化验证的一种形式，虽能提供比静态分析器更精确的结果，却需要大量资源并显著阻碍开发者的工作效率。机器学习能否达到与形式化验证方法相当的准确性，并在流行的即时代码补全框架中以近实时方式应用？本文提出SecureFalcon——一种创新的模型架构，其仅包含1.21亿参数，源自Falcon-40B模型并专门针对软件漏洞分类任务定制。为获得最佳性能，我们使用两个数据集训练模型：FormAI数据集和FalconVulnDB。FalconVulnDB整合了近期公开数据集，包括SySeVR框架、Draper VDISC、Bigvul、Diversevul、SARD Juliet和ReVeal数据集。这些数据集涵盖前25种最危险的软件缺陷类型，例如CWE-119、CWE-120、CWE-476、CWE-122、CWE-190、CWE-121、CWE-78、CWE-787、CWE-20和CWE-762。SecureFalcon在二元分类任务中达到94%的准确率，在多分类任务中最高达92%，且支持即时CPU推理。其性能超越现有模型（如BERT、RoBERTa、CodeBERT）及传统机器学习算法，有望推动软件漏洞检测与即时代码补全框架的发展边界。