Achieving reproducibility, quantity, and diversity in vulnerability datasets has long been viewed as an inherent three-way trade-off, where improving one dimension often comes at the cost of the others. In practice, reproducibility has been the dimension most often neglected. This has limited what can be automatically extracted from historical bug datasets, and has reduced their utility for downstream security research. In this work, we propose a method to produce a new security dataset which ensures reproducibility for diverse vulnerabilities at scale by identifying the key obstacles to large-scale bug reproduction and addressing them with general solutions. Using this method, we introduce full reproducibility to the largest open source software vulnerability dataset (OSS-Fuzz) and construct the ARVO dataset (an Atlas of Reproducible Vulnerabilities in Open-source software). ARVO is a large-scale dataset consisting of over 6,100 real-world vulnerabilities across 311 projects. Focusing on reproducibility, ARVO differs from existing datasets by providing each vulnerability in a form that can be consistently rebuilt, triggered, and analyzed across versions. Reproducibility also enables automatic identification of the corresponding patch for each vulnerability and supports direct interaction with vulnerabilities after code changes, capabilities that existing large-scale datasets do not provide. In our evaluation, ARVO successfully reproduces 81% of vulnerabilities and achieves 89.4% accuracy on the located patches. We also discuss ARVO's influence on both upstream practices and downstream security research.

翻译：长期以来，实现漏洞数据集的可复现性、充足数量和多样性被视为固有的三元权衡关系：提升某一维度往往以牺牲其他维度为代价。实践中，可复现性常成为最被忽视的维度。这限制了从历史漏洞数据集中自动提取信息的能力，并削弱了其在安全下游研究中的实用价值。本文提出一种方法，通过识别大规模漏洞复现的关键障碍并采用通用解决方案加以应对，从而构建一个确保多样漏洞在大规模范围内具备可复现性的新型安全数据集。基于该方法，我们为最大规模的开源软件漏洞数据集（OSS-Fuzz）引入了完全可复现性，并由此构建了ARVO数据集（开源软件可复现漏洞图谱）。ARVO是一个大规模数据集，涵盖311个项目中的6100余个真实世界漏洞。聚焦可复现性，ARVO区别于现有数据集之处在于：每个漏洞均以可在版本间持续重建、触发和分析的形式提供。可复现性还支持自动识别每个漏洞的对应补丁，并在代码变更后直接与漏洞交互——这是现有大规模数据集所不具备的能力。评估显示，ARVO成功复现了81%的漏洞，并实现了89.4%的补丁定位准确率。我们还将探讨ARVO对上游实践与下游安全研究的双重影响。