We investigated the capabilities of GPT-4o and Gemini 2.0 Flash for secure Infrastructure as Code (IaC) development. For security smell detection, on the Stack Overflow dataset, which primarily contains small, simplified code snippets, the models detected at least 71% of security smells when prompted to analyze code from a security perspective (general prompt). With a guided prompt (adding clear, step-by-step instructions), this increased to 78%.In GitHub repositories, which contain complete, real-world project scripts, a general prompt was less effective, leaving more than half of the smells undetected. However, with the guided prompt, the models uncovered at least 67% of the smells. For secure code generation, we prompted LLMs with 89 vulnerable synthetic scenarios and observed that only 7% of the generated scripts were secure. Adding an explicit instruction to generate secure code increased GPT secure output rate to 17%, while Gemini changed little (8%). These results highlight the need for further research to improve LLMs' capabilities in assisting developers with secure IaC development.

翻译：本研究评估了GPT-4o与Gemini 2.0 Flash在安全基础设施即代码（IaC）开发中的能力。在安全异味检测方面，针对主要包含小型简化代码片段的Stack Overflow数据集，当提示模型从安全角度分析代码时（通用提示），模型至少能检测出71%的安全异味；采用引导式提示（添加清晰的分步指令）后，检测率提升至78%。在包含完整实际项目脚本的GitHub仓库中，通用提示效果较差，超过半数的异味未被发现；而引导式提示使模型至少能发现67%的异味。在安全代码生成方面，我们通过89个存在漏洞的合成场景提示大语言模型，发现仅有7%的生成脚本是安全的。添加明确的安全编码指令后，GPT的安全输出率提升至17%，而Gemini改进有限（8%）。这些结果表明，需要进一步研究以提升大语言模型辅助开发者进行安全IaC开发的能力。