This paper performs the first study to understand the prevalence, challenges, and effectiveness of using Static Application Security Testing (SAST) tools on Open-Source Embedded Software (EMBOSS) repositories. We collect a corpus of 258 of the most popular EMBOSS projects, representing 13 distinct categories such as real-time operating systems, network stacks, and applications. To understand the current use of SAST tools on EMBOSS, we measured this corpus and surveyed developers. To understand the challenges and effectiveness of using SAST tools on EMBOSS projects, we applied these tools to the projects in our corpus. We report that almost none of these projects (just 3%) use SAST tools beyond those baked into the compiler, and developers give rationales such as ineffectiveness and false positives. In applying SAST tools ourselves, we show that minimal engineering effort and project expertise are needed to apply many tools to a given EMBOSS project. GitHub's CodeQL was the most effective SAST tool -- using its built-in security checks we found a total of 540 defects (with a false positive rate of 23%) across the 258 projects, with 399 (74%) likely security vulnerabilities, including in projects maintained by Microsoft, Amazon, and the Apache Foundation. EMBOSS engineers have confirmed 273 (51%) of these defects, mainly by accepting our pull requests. Two CVEs were issued. In summary, we urge EMBOSS engineers to adopt the current generation of SAST tools, which offer low false positive rates and are effective at finding security-relevant defects.

翻译：本文首次系统研究了静态应用安全测试（SAST）工具在开源嵌入式软件（EMBOSS）仓库中的普及程度、挑战与有效性。我们收集了258个最热门的EMBOSS项目语料库，涵盖实时操作系统、网络协议栈和应用程序等13个不同类别。为探究SAST工具在EMBOSS中的当前应用情况，我们对语料库进行量化分析并开展开发者调研；为理解SAST工具应用于EMBOSS项目时面临的挑战与效能，我们将这些工具实际应用于语料库中的项目。研究发现，仅3%的项目使用了编译器内置工具之外的SAST工具，开发者给出的理由包括工具效果不佳和误报率过高。通过自主应用SAST工具，我们证明在给定EMBOSS项目上部署多种工具仅需极少的工程投入与项目专业知识。GitHub的CodeQL是最有效的SAST工具——利用其内置安全检查，我们在258个项目中总计发现540个缺陷（误报率23%），其中399个（74%）可能为安全漏洞，涉及微软、亚马逊和阿帕奇基金会维护的项目。EMBOSS工程师已确认273个（51%）缺陷，主要通过接受我们的拉取请求达成验证。两个漏洞已获发CVE编号。综上，我们敦促EMBOSS工程师采用当前一代SAST工具，这类工具误报率低且能有效发现安全相关缺陷。