The recent improvement in code generation capabilities due to the use of large language models has mainly benefited general purpose programming languages. Domain specific languages, such as the ones used for IT Automation, have received far less attention, despite involving many active developers and being an essential component of modern cloud platforms. This work focuses on the generation of Ansible-YAML, a widely used markup language for IT Automation. We present Ansible Wisdom, a natural-language to Ansible-YAML code generation tool, aimed at improving IT automation productivity. Ansible Wisdom is a transformer-based model, extended by training with a new dataset containing Ansible-YAML. We also develop two novel performance metrics for YAML and Ansible to capture the specific characteristics of this domain. Results show that Ansible Wisdom can accurately generate Ansible script from natural language prompts with performance comparable or better than existing state of the art code generation models. In few-shot settings we asses the impact of training with Ansible, YAML data and compare with different baselines including Codex-Davinci-002. We also show that after finetuning, our Ansible specific model can beat the performance of a much larger Codex-Davinci-002 in few shot settings.

翻译：近期由于大型语言模型的应用，代码生成能力的提升主要惠及通用编程语言。领域特定语言（如IT自动化所使用的语言）虽涉及众多活跃开发者且是现代云平台的核心组件，却受到较少关注。本研究聚焦于Ansible-YAML的生成——这是一种广泛用于IT自动化的标记语言。我们提出Ansible Wisdom，一个面向自然语言到Ansible-YAML代码生成的工具，旨在提升IT自动化效率。Ansible Wisdom基于Transformer模型，通过包含Ansible-YAML的新数据集进行扩展训练。我们同时针对YAML和Ansible领域特性开发了两种新型性能评估指标。实验表明，Ansible Wisdom能从自然语言提示中准确生成Ansible脚本，其性能与现有最先进代码生成模型相当或更优。在少样本学习场景中，我们评估了使用Ansible、YAML数据训练的模型性能，并与包括Codex-Davinci-002在内的多个基线模型进行对比。此外，经微调后，我们的Ansible专用模型在少样本环境下可超越规模更大的Codex-Davinci-002模型性能。