Recently, deep learning techniques have garnered substantial attention for their ability to identify vulnerable code patterns accurately. However, current state-of-the-art deep learning models, such as Convolutional Neural Networks (CNN), and Long Short-Term Memories (LSTMs) require substantial computational resources. This results in a level of overhead that makes their implementation unfeasible for deployment in realtime settings. This study presents a novel transformer-based vulnerability detection framework, referred to as VulDetect, which is achieved through the fine-tuning of a pre-trained large language model, (GPT) on various benchmark datasets of vulnerable code. Our empirical findings indicate that our framework is capable of identifying vulnerable software code with an accuracy of up to 92.65%. Our proposed technique outperforms SyseVR and VulDeBERT, two state-of-the-art vulnerability detection techniques

翻译：近年来，深度学习技术因其能够准确识别易受攻击的代码模式而备受关注。然而，当前最先进的深度学习模型，例如卷积神经网络（CNN）和长短期记忆网络（LSTM），需要大量的计算资源。这导致了较高的开销，使其无法在实时环境中部署。本研究提出了一种新颖的基于Transformer的漏洞检测框架，称为VulDetect，通过在各种易受攻击代码的基准数据集上微调预训练的大型语言模型（GPT）来实现。我们的实验结果表明，该框架能够以高达92.65%的准确率识别易受攻击的软件代码。我们提出的技术优于两种最先进的漏洞检测技术——SyseVR和VulDeBERT。