GitHub Actions is a widely used platform that allows developers to automate the build and deployment of their projects through configurable workflows. As the platform's popularity continues to grow, it has become a target of choice for recent software supply chain attacks. These attacks exploit excessive permissions, ambiguous versions, or the absence of artifact integrity checks to compromise workflows. In response to these attacks, several security scanners have emerged to help developers harden their workflows. In this paper, we perform the first systematic comparison of 9 GitHub Actions workflow security scanners. We compare them in terms of scope (which security weaknesses they target), detection capabilities (how many weaknesses they detect), and usability (how long they take to scan a workflow). To compare scanners on a common ground, we first establish a taxonomy of 10 security weaknesses that can occur in GitHub Actions workflows. Then, we run the scanners against a curated set of 596 workflows. Our study reveals that the landscape of GitHub Actions workflow security scanners is diverse, with both broad-scope tools and very focused ones. More importantly, we show that scanners interpret security weaknesses differently, leading to significant differences in the type and number of reported weaknesses. Based on this empirical evidence, we make actionable recommendations for developers to harden their GitHub Actions workflows.

翻译：GitHub Actions是一个广泛使用的平台，允许开发者通过可配置的工作流实现项目构建与部署的自动化。随着该平台持续普及，其已成为近期软件供应链攻击的首选目标。此类攻击通过利用过度权限、模糊版本或缺乏制品完整性检查等手段来入侵工作流。为应对这些攻击，多种安全扫描工具应运而生，以帮助开发者加固其工作流。本文首次对9种GitHub Actions工作流安全扫描工具进行了系统性比较，从覆盖范围（针对的安全弱点类型）、检测能力（可识别的弱点数量）和可用性（扫描工作流所需时间）三个维度展开评估。为建立统一的比较基准，我们首先构建了包含10类GitHub Actions工作流安全弱点的分类体系，随后使用精选的596个工作流样本对各扫描工具进行测试。研究表明，当前GitHub Actions工作流安全扫描工具生态呈现多样化特征，既存在覆盖广泛的工具，也存在高度聚焦的专用工具。更重要的是，我们发现不同扫描工具对安全弱点的判定标准存在差异，导致其报告弱点的类型和数量存在显著区别。基于实证证据，我们为开发者加固GitHub Actions工作流提出了可操作的建议。