Despite their remarkable success, large language models (LLMs) have shown limited ability on applied tasks such as vulnerability detection. We investigate various prompting strategies for vulnerability detection and, as part of this exploration, propose a prompting strategy that integrates natural language descriptions of vulnerabilities with a contrastive chain-of-thought reasoning approach, augmented using contrastive samples from a synthetic dataset. Our study highlights the potential of LLMs to detect vulnerabilities by integrating natural language descriptions, contrastive reasoning, and synthetic examples into a comprehensive prompting framework. Our results show that this approach can enhance LLM understanding of vulnerabilities. On a high-quality vulnerability detection dataset such as SVEN, our prompting strategies can improve accuracies, F1-scores, and pairwise accuracies by 23%, 11%, and 14%, respectively.

翻译：尽管大型语言模型（LLM）取得了显著成功，但在漏洞检测等应用任务上表现有限。我们研究了多种用于漏洞检测的提示策略，并在此探索过程中提出了一种提示策略，该策略将漏洞的自然语言描述与对比性思维链推理方法相结合，并利用来自合成数据集的对比样本进行增强。我们的研究凸显了LLM通过将自然语言描述、对比推理和合成示例整合到一个全面的提示框架中来检测漏洞的潜力。结果表明，该方法能增强LLM对漏洞的理解。在SVEN等高质漏洞检测数据集上，我们的提示策略可将准确率、F1分数和成对准确率分别提升23%、11%和14%。