The Vulnerability Exploitability eXchange (VEX) format has been introduced to complement Software Bill of Materials (SBOM) with security advisories of known vulnerabilities. VEX gives an accurate understanding of vulnerabilities found in the dependencies of third-party software, which is critical for secure software development and risk analysis. In this paper, we present a study that analyzes state-of-the-art VEX-generation tools (Trivy, Grype, DepScan, Scout, Snyk, OSV, Vexy) applied to containers. Our study examines how consistently different VEX-generation tools perform. By evaluating their performance across multiple datasets, we aim to gain insight into the overall maturity of the VEX-generation tool ecosystem, beyond any single implementation. We use the Jaccard and Tversky indices to produce similarity scores of tool results for three different datasets created from container images. Overall, our results show a low level of consistency among the tools, thus indicating a low level of maturity in the VEX tool space. We perform a number of experiments to explore the impact of different factors on the consistency of the results, with the difference in vulnerability databases queried showing the largest impact.
翻译:漏洞可利用性交换(VEX)格式已被引入,旨在通过已知漏洞的安全公告来补充软件物料清单(SBOM)。VEX能够准确理解第三方软件依赖项中存在的漏洞,这对于安全的软件开发与风险分析至关重要。本文针对应用于容器的现有VEX生成工具(Trivy, Grype, DepScan, Scout, Snyk, OSV, Vexy)开展研究,分析这些工具在生成结果上的一致性表现。通过评估其在多个数据集上的性能,我们旨在深入了解VEX生成工具生态系统超越单一实现后的整体成熟度。我们采用Jaccard指数和Tversky指数,对基于容器镜像构建的三个不同数据集所得工具结果计算相似度评分。总体而言,研究结果表明各工具之间的一致性较低,反映出VEX工具领域整体成熟度不足。我们实施了一系列实验以探究不同因素对结果一致性的影响,其中所查询的漏洞数据库差异是影响最大的因素。