Context: Static analyses are well-established to aid in understanding bugs or vulnerabilities during the development process or in large-scale studies. A low false-positive rate is essential for the adaption in practice and for precise results of empirical studies. Unfortunately, static analyses tend to report where a vulnerability manifests rather than the fix location. This can cause presumed false positives or imprecise results. Method: To address this problem, we designed an adaption of an existing static analysis algorithm that can distinguish between a manifestation and fix location, and reports error chains. An error chain represents at least two interconnected errors that occur successively, thus building the connection between the fix and manifestation location. We used our tool CogniCryptSUBS for a case study on 471 GitHub repositories, a performance benchmark to compare different analysis configurations, and conducted an expert interview. Result: We found that 50 % of the projects with a report had at least one error chain. Our runtime benchmark demonstrated that our improvement caused only a minimal runtime overhead of less than 4 %. The results of our expert interview indicate that with our adapted version participants require fewer executions of the analysis. Conclusion: Our results indicate that error chains occur frequently in real-world projects, and ignoring them can lead to imprecise evaluation results. The runtime benchmark indicates that our tool is a feasible and efficient solution for detecting error chains in real-world projects. Further, our results gave a hint that the usability of static analyses may benefit from supporting error chains.
翻译:背景:静态分析在开发过程或大规模研究中已被广泛用于辅助理解缺陷或漏洞。低误报率对于实践应用和实证研究的精确结果至关重要。然而,静态分析通常报告漏洞的表现位置而非修复位置,这可能导致误判的误报或不精确的结果。方法:为解决该问题,我们设计了一种现有静态分析算法的改编版本,能够区分表现位置与修复位置,并报告错误链。错误链代表至少两个连续发生的互连错误,从而构建修复位置与表现位置之间的关联。我们使用工具CogniCryptSUBS对471个GitHub仓库进行案例研究、性能基准测试以比较不同分析配置,并开展了专家访谈。结果:我们发现50%存在报告的项目至少包含一个错误链。运行时基准测试表明,我们的改进仅带来低于4%的极低运行时开销。专家访谈结果表明,使用改编版本后,参与者所需的分析执行次数更少。结论:研究结果表明,错误链在实际项目中频繁出现,忽略它们可能导致不精确的评估结果。运行时基准测试表明,我们的工具是检测实际项目中错误链的可行且高效的解决方案。此外,结果提示支持错误链可能提升静态分析的可用性。