This paper presents a study that analyzed state-of-the-art vulnerability scanning tools applied to containers. We have focused the work on tools following the Vulnerability Exploitability eXchange (VEX) format, which has been introduced to complement Software Bills of Material (SBOM) with security advisories of known vulnerabilities. Being able to get an accurate understanding of vulnerabilities found in the dependencies of third-party software is critical for secure software development and risk analysis. Accepting the overwhelming challenge of estimating the precise accuracy and precision of a vulnerability scanner, we have in this study instead set out to explore how consistently different tools perform. By doing this, we aim to assess the maturity of the VEX tool field as a whole (rather than any particular tool). We have used the Jaccard and Tversky indices to produce similarity scores of tool performance for several different datasets created from container images. Overall, our results show a low level of consistency among the tools, thus indicating a low level of maturity in VEX tool space. We have performed a number of experiments to find and explanation to our results, but largely they are inconclusive and further research is needed to understand the underlying causalities of our findings.

翻译：本文对应用于容器的最新漏洞扫描工具进行了分析研究。我们重点关注遵循漏洞可利用性交换（VEX）格式的工具，该格式旨在通过已知漏洞的安全公告来补充软件物料清单（SBOM）。准确理解第三方软件依赖项中发现的漏洞，对于安全的软件开发与风险分析至关重要。鉴于准确评估漏洞扫描器精确度与准确度面临巨大挑战，本研究转而探索不同工具间性能的一致性。通过此方法，我们旨在评估整个VEX工具领域的成熟度（而非特定工具）。我们采用Jaccard指数和Tversky指数，基于容器镜像构建的多个数据集，对工具性能的相似性进行评分。总体而言，结果显示工具间一致性较低，表明VEX工具领域成熟度不足。尽管我们开展了多项实验以解释研究结果，但结论大多不明确，需要进一步研究以理解这些发现背后的根本原因。