Large language models (LLMs) have been proposed as powerful tools for detecting software vulnerabilities, where task-specific fine-tuning is typically employed to provide vulnerability-specific knowledge to the LLMs for this purpose. However, traditional full-parameter fine-tuning is inefficient for modern, complex LLMs, which contain billions of parameters. Soft prompt tuning has been suggested as a more efficient alternative for fine-tuning LLMs in general cases. However, pure soft prompt tuning treats source code as plain text, losing structural information inherent in source code. Meanwhile, graph-enhanced soft prompt tuning methods, which aim to address this issue, are unable to preserve the rich semantic information within code graphs, as they are primarily designed for general graph-related tasks and focus more on adjacency information. They also fail to ensure computational efficiency while accounting for graph-text interactions. This paper, therefore, introduces a new code graph-enhanced, structure-aware soft prompt tuning method for vulnerability detection, referred to as CGP-Tuning. It employs innovative type-aware embeddings to capture the rich semantic information within code graphs, along with a novel and efficient cross-modal alignment module that achieves linear computational cost while incorporating graph-text interactions. The proposed CGP-Tuning is evaluated on the latest DiverseVul dataset and the most recent open-source code LLMs, CodeLlama and CodeGemma. Experimental results demonstrate that CGP-Tuning outperforms the best state-of-the-art method by an average of 3.5 percentage points in accuracy, without compromising its vulnerability detection capabilities for long source code.

翻译：大型语言模型（LLM）已被提出作为检测软件漏洞的强大工具，为此通常采用任务特定的微调来为LLM提供漏洞相关知识。然而，对于包含数十亿参数的现代复杂LLM，传统的全参数微调效率低下。软提示调优在一般情况下被建议作为微调LLM的更高效替代方案。然而，纯软提示调优将源代码视为纯文本，丢失了源代码固有的结构信息。同时，旨在解决此问题的图增强软提示调优方法，由于主要针对通用图相关任务且更侧重于邻接信息，无法保留代码图中丰富的语义信息。这些方法在考虑图-文本交互时也未能保证计算效率。因此，本文提出了一种新的用于漏洞检测的代码图增强、结构感知软提示调优方法，称为CGP-Tuning。该方法采用创新的类型感知嵌入来捕获代码图中丰富的语义信息，并结合一个新颖高效的多模态对齐模块，该模块在融入图-文本交互的同时实现了线性计算成本。所提出的CGP-Tuning在最新的DiverseVul数据集以及当前最先进的开源代码LLM（CodeLlama和CodeGemma）上进行了评估。实验结果表明，CGP-Tuning在准确率上平均优于最佳现有方法3.5个百分点，且未损害其对长源代码的漏洞检测能力。