The Proof-of-Concept (PoC) for a vulnerability is crucial in validating its existence, mitigating false positives, and illustrating the severity of the security threat it poses. However, research on PoCs significantly lags behind studies focusing on vulnerability data. This discrepancy can be directly attributed to several challenges, including the dispersion of real-world PoCs across multiple platforms, the diversity in writing styles, and the difficulty associated with PoC reproduction. To fill this gap, we conduct the first large-scale study on PoCs in the wild, assessing their report availability, completeness, reproducibility. Specifically, 1) to investigate PoC reports availability for CVE vulnerability, we collected an extensive dataset of 470,921 PoCs and their reports from 13 platforms, representing the broadest collection of publicly available PoCs to date. 2) To assess the completeness of PoC report at a fine-grained level, we proposed a component extraction method, which combines pattern-matching techniques with a fine-tuned BERT-NER model to extract 9 key components from PoC reports. 3) To evaluate the effectiveness of PoCs, we recruited 8 participants to manually reproduce 150 sampled vulnerabilities with 32 vulnerability types based on PoC reports, enabling an in-depth analysis of PoC reproducibility and the factors influencing it. Our findings reveal that 78.9% of CVE vulnerabilities lack available PoCs, and existing PoC reports typically miss about 30% of the essential components required for effective vulnerability understanding and reproduction, with various reasons identified for the failure to reproduce vulnerabilities using available PoC reports. Finally, we proposed actionable strategies for stakeholders to enhance the overall usability of vulnerability PoCs in strengthening software security.

翻译：漏洞的概念验证（PoC）对于验证漏洞存在性、减少误报以及阐明其安全威胁的严重性至关重要。然而，针对PoC的研究显著滞后于对漏洞数据的研究。这种差距可直接归因于若干挑战，包括真实世界PoC分散于多个平台、编写风格多样性以及与PoC复现相关的困难。为填补这一空白，我们开展了首次针对真实环境中PoC的大规模研究，评估其报告可用性、完整性和可复现性。具体而言：1）为调查CVE漏洞的PoC报告可用性，我们从13个平台收集了包含470,921个PoC及其报告的广泛数据集，这是迄今为止最全面的公开可用PoC集合。2）为在细粒度层面评估PoC报告的完整性，我们提出了一种组件提取方法，该方法结合模式匹配技术与微调的BERT-NER模型，从PoC报告中提取9个关键组件。3）为评估PoC的有效性，我们招募了8名参与者，基于PoC报告手动复现了涵盖32种漏洞类型的150个抽样漏洞，从而深入分析PoC的可复现性及其影响因素。我们的研究结果表明：78.9%的CVE漏洞缺乏可用PoC；现有PoC报告平均缺失约30%理解与复现漏洞所需的关键组件；同时我们识别出导致利用现有PoC报告复现漏洞失败的多种原因。最后，我们为相关利益方提出了可操作的改进策略，以提升漏洞PoC的整体可用性，从而加强软件安全性。