Deep Learning (DL)-based methods have proven to be effective for software vulnerability detection, with a potential for substantial productivity enhancements for detecting vulnerabilities. Current methods mainly focus on detecting single functions (i.e., intra-procedural vulnerabilities), ignoring the more complex inter-procedural vulnerability detection scenarios in practice. For example, developers routinely engage with program analysis to detect vulnerabilities that span multiple functions within repositories. In addition, the widely-used benchmark datasets generally contain only intra-procedural vulnerabilities, leaving the assessment of inter-procedural vulnerability detection capabilities unexplored. To mitigate the issues, we propose a repository-level evaluation system, named \textbf{VulEval}, aiming at evaluating the detection performance of inter- and intra-procedural vulnerabilities simultaneously. Specifically, VulEval consists of three interconnected evaluation tasks: \textbf{(1) Function-Level Vulnerability Detection}, aiming at detecting intra-procedural vulnerability given a code snippet; \textbf{(2) Vulnerability-Related Dependency Prediction}, aiming at retrieving the most relevant dependencies from call graphs for providing developers with explanations about the vulnerabilities; and \textbf{(3) Repository-Level Vulnerability Detection}, aiming at detecting inter-procedural vulnerabilities by combining with the dependencies identified in the second task. VulEval also consists of a large-scale dataset, with a total of 4,196 CVE entries, 232,239 functions, and corresponding 4,699 repository-level source code in C/C++ programming languages. Our analysis highlights the current progress and future directions for software vulnerability detection.

翻译：基于深度学习的方法已被证明在软件漏洞检测中有效，并有可能显著提升检测效率。当前方法主要聚焦于单函数检测（即过程内漏洞），忽略了实践中更为复杂的过程间漏洞检测场景。例如，开发人员通常需要借助程序分析来检测仓库中跨多个函数的漏洞。此外，广泛使用的基准数据集通常仅包含过程内漏洞，导致过程间漏洞检测能力的评估尚未得到探索。为解决这些问题，我们提出了一个名为**VulEval**的仓库级评估系统，旨在同时评估过程间与过程内漏洞的检测性能。具体而言，VulEval包含三个相互关联的评估任务：**（1）函数级漏洞检测**，旨在检测给定代码片段中的过程内漏洞；**（2）漏洞相关依赖预测**，旨在从调用图中检索最相关的依赖关系，为开发者提供漏洞解释；**（3）仓库级漏洞检测**，通过结合第二项任务识别出的依赖关系，检测过程间漏洞。VulEval还包含一个大规模数据集，总计包含4,196个CVE条目、232,239个函数以及对应的4,699个C/C++编程语言的仓库级源代码。我们的分析揭示了软件漏洞检测的当前进展与未来方向。