The application of Machine Learning techniques in code generation is now a common practice for most developers. Tools such as ChatGPT from OpenAI leverage the natural language processing capabilities of Large Language Models to generate machine code from natural language descriptions. In the cybersecurity field, red teams can also take advantage of generative models to build malicious code generators, providing more automation to Pentest audits. However, the application of Large Language Models in malicious code generation remains challenging due to the lack of data to train and evaluate offensive code generators. In this work, we propose RedShell, a tool that allows ethical hackers to generate malicious PowerShell code. We also introduce a ground truth dataset, combining publicly available code samples to fine-tune models in malicious PowerShell generation. Our experiments demonstrate the strong capabilities of RedShell in generating syntactically valid PowerShell, with fewer than 10% of the generated samples resulting in parse errors. Furthermore, our specialized model was able to produce samples that were semantically consistent with reference snippets, achieving a competitive performance on standard output similarity metrics such as Edit Distance and METEOR, with their mean similarity scores exceeding 50% and 40%, respectively. This work sheds light on the state-of-the-art research in the field of Generative AI applied to Pentesting, and also serves as a steppingstone for future advancements, highlighting the potential benefits these models hold within such controlled environments.

翻译：机器学习技术在代码生成中的应用如今已成为大多数开发者的普遍实践。诸如OpenAI的ChatGPT等工具利用大语言模型的自然语言处理能力，从自然语言描述中生成机器代码。在网络安全领域，红队亦可利用生成式模型构建恶意代码生成器，从而为渗透测试审计提供更高程度的自动化。然而，由于缺乏训练和评估恶意代码生成器的数据，大语言模型在恶意代码生成中的应用仍面临挑战。本文提出RedShell这一工具，使道德黑客能够生成恶意的PowerShell代码。我们还引入了一个真实数据集，结合公开可用的代码样本，以微调恶意PowerShell生成的模型。实验表明，RedShell在生成语法有效的PowerShell代码方面表现强劲，生成的样本中语法错误比例低于10%。此外，我们的专用模型能够生成与参考片段语义一致的样本，在编辑距离和METEOR等标准输出相似度指标上取得了有竞争力的表现，其平均相似度得分分别超过50%和40%。本研究揭示了生成式人工智能应用于渗透测试领域的前沿研究，并作为未来进展的基石，突出了此类模型在受控环境中的潜在优势。