GitHub Actions is a widely used platform to automate the build and deployment of software projects through configurable workflows. As the platform's popularity grows, it also becomes a target of choice for software supply chain attacks. These attacks exploit excessive permissions, ambiguous versions or the absence of artifact integrity checks to compromise the workflows. In response to these attacks, several security scanners have emerged to help developers harden their workflows. In this paper, we perform the first systematic comparison of 9 GitHub Actions Workflows security scanners. We compare them regarding scope (which security weaknesses they target), detection capabilities (how many weaknesses they detect), and performance (how long they take to scan a workflow). In order to compare the scanners on a common ground, we first establish a classification of 10 common security weaknesses that can be found in GitHub Actions Workflows. Then, we run the scanners against a curated set of 2722 workflows. Our study reveals that the landscape of GitHub Actions Workflows security scanners is very diverse, with both general purpose and focused scanners. More importantly, we provide evidence that these scanners implement fundamentally different analysis strategies, leading to major gaps regarding the nature and the number of reported security weaknesses. Based on these empirical evidence we make actionable recommendations for developers to harden their GitHub Actions Workflows.

翻译：GitHub Actions是一个广泛使用的平台，通过可配置的工作流实现软件项目的自动化构建与部署。随着该平台普及度的提升，其也成为软件供应链攻击的重要目标。此类攻击常通过滥用过度权限、利用版本模糊性或绕过制品完整性检查等手段危害工作流安全。为应对这些威胁，目前已涌现出多款帮助开发者加固工作流的安全扫描工具。本文首次对9款GitHub Actions工作流安全扫描工具进行了系统性比较，从检测范围（针对哪些安全弱点）、检测能力（可识别弱点数量）和运行性能（扫描工作流耗时）三个维度展开评估。为建立统一的比较基准，我们首先对GitHub Actions工作流中常见的10类安全弱点进行了系统分类，随后使用精心筛选的2722个工作流样本对扫描工具进行测试。研究表明，当前GitHub Actions工作流安全扫描工具生态呈现高度多样化特征，既存在通用型扫描器，也包含专用型工具。更重要的是，我们发现这些扫描工具采用了根本不同的分析策略，导致其在报告安全弱点的类型和数量方面存在显著差异。基于实证研究结果，我们为开发者提供了可操作的GitHub Actions工作流加固建议。