Automatic detection of software bugs is a critical task in software security. Many static tools that can help detect bugs have been proposed. While these static bug detectors are mainly evaluated on general software projects call into question their practical effectiveness and usefulness for machine learning libraries. In this paper, we address this question by analyzing five popular and widely used static bug detectors, i.e., Flawfinder, RATS, Cppcheck, Facebook Infer, and Clang static analyzer on a curated dataset of software bugs gathered from four popular machine learning libraries including Mlpack, MXNet, PyTorch, and TensorFlow with a total of 410 known bugs. Our research provides a categorization of these tools' capabilities to better understand the strengths and weaknesses of the tools for detecting software bugs in machine learning libraries. Overall, our study shows that static bug detectors find a negligible amount of all bugs accounting for 6/410 bugs (0.01%), Flawfinder and RATS are the most effective static checker for finding software bugs in machine learning libraries. Based on our observations, we further identify and discuss opportunities to make the tools more effective and practical.

翻译：自动检测软件缺陷是软件安全领域中的一项关键任务。目前已提出多种能够辅助检测缺陷的静态工具。然而，这些静态缺陷检测器主要针对通用软件项目进行评估，这引发了对其在机器学习库中实际有效性和可用性的质疑。本文通过分析五种广泛使用的静态缺陷检测器（Flawfinder、RATS、Cppcheck、Facebook Infer和Clang静态分析器），基于从四个流行机器学习库（包括Mlpack、MXNet、PyTorch和TensorFlow）中收集的410个已知软件缺陷的精心整理数据集，旨在解答上述问题。我们对这些工具的能力进行了分类，以更深入地理解它们在检测机器学习库中软件缺陷时的优势与不足。总体而言，研究表明静态缺陷检测器仅能发现极少数缺陷（6/410，占比0.01%），其中Flawfinder和RATS是检测机器学习库软件缺陷最有效的静态检查工具。基于观察结果，我们进一步识别并探讨了提升这些工具实用性和有效性的机遇。