Accompanying the successes of learning-based defensive software vulnerability analyses is the lack of large and quality sets of labeled vulnerable program samples, which impedes further advancement of those defenses. Existing automated sample generation approaches have shown potentials yet still fall short of practical expectations due to the high noise in the generated samples. This paper proposes VGX, a new technique aimed for large-scale generation of high-quality vulnerability datasets. Given a normal program, VGX identifies the code contexts in which vulnerabilities can be injected, using a customized Transformer featured with a new value-flowbased position encoding and pre-trained against new objectives particularly for learning code structure and context. Then, VGX materializes vulnerability-injection code editing in the identified contexts using patterns of such edits obtained from both historical fixes and human knowledge about real-world vulnerabilities. Compared to four state-of-the-art (SOTA) baselines (pattern-, Transformer-, GNN-, and pattern+Transformer-based), VGX achieved 99.09-890.06% higher F1 and 22.45%-328.47% higher label accuracy. For in-the-wild sample production, VGX generated 150,392 vulnerable samples, from which we randomly chose 10% to assess how much these samples help vulnerability detection, localization, and repair. Our results show SOTA techniques for these three application tasks achieved 19.15-330.80% higher F1, 12.86-19.31% higher top-10 accuracy, and 85.02-99.30% higher top-50 accuracy, respectively, by adding those samples to their original training data. These samples also helped a SOTA vulnerability detector discover 13 more real-world vulnerabilities (CVEs) in critical systems (e.g., Linux kernel) that would be missed by the original model.

翻译：伴随基于学习的防御性软件漏洞分析取得成功，高质量标注漏洞程序样本的缺乏正阻碍这些防御技术的进一步发展。现有自动化样本生成方法虽展现出潜力，但因生成样本中存在的严重噪声而仍难以达到实际应用期望。本文提出VGX——一种旨在大规模生成高质量漏洞数据集的新技术。给定正常程序，VGX通过定制化Transformer识别可注入漏洞的代码上下文，该Transformer采用新型基于值流的位置编码，并通过针对代码结构与上下文学习的全新预训练目标进行优化。随后，VGX在已识别的上下文中，利用从历史修复记录及真实世界漏洞人类知识中提取的漏洞注入模式，具体实现漏洞注入代码编辑。与四种最先进基线方法（基于模式、Transformer、GNN及模式+Transformer）相比，VGX的F1值提升99.09%-890.06%，标签准确率提升22.45%-328.47%。在实际样本生成中，VGX产出150,392个漏洞样本，我们随机选取其中10%评估其对漏洞检测、定位及修复任务的助益。实验表明，将这些样本加入原始训练数据后，三种下游任务的SOTA技术在F1值、Top-10准确率及Top-50准确率上分别提升19.15%-330.80%、12.86%-19.31%及85.02%-99.30%。此外，这些样本帮助一种SOTA漏洞检测器在关键系统（如Linux内核）中发现13个原始模型遗漏的真实世界漏洞（CVE）。