Translating natural language into Bash Commands is an emerging research field that has gained attention in recent years. Most efforts have focused on producing more accurate translation models. To the best of our knowledge, only two datasets are available, with one based on the other. Both datasets involve scraping through known data sources (through platforms like stack overflow, crowdsourcing, etc.) and hiring experts to validate and correct either the English text or Bash Commands. This paper provides two contributions to research on synthesizing Bash Commands from scratch. First, we describe a state-of-the-art translation model used to generate Bash Commands from the corresponding English text. Second, we introduce a new NL2CMD dataset that is automatically generated, involves minimal human intervention, and is over six times larger than prior datasets. Since the generation pipeline does not rely on existing Bash Commands, the distribution and types of commands can be custom adjusted. We evaluate the performance of ChatGPT on this task and discuss the potential of using it as a data generator. Our empirical results show how the scale and diversity of our dataset can offer unique opportunities for semantic parsing researchers.
翻译:将自然语言翻译为Bash命令是一个近年来受到关注的新兴研究领域。多数工作聚焦于提升翻译模型的精度。据我们所知,当前仅有两种数据集可用,且其中一种基于另一种构建。这两种数据集均需通过已知数据源(如Stack Overflow、众包等平台)采集信息,并聘请专家对英文文本或Bash命令进行验证与修正。本文为从零合成Bash命令的研究做出了两项贡献:首先,我们描述了一种用于从对应英文文本生成Bash命令的最先进翻译模型;其次,我们提出了一个新型NL2CMD数据集,该数据集自动生成、人工干预极少,且规模是先前数据集的六倍以上。由于生成流程不依赖现有Bash命令,可自定义调整命令的分布与类型。我们评估了ChatGPT在此任务上的表现,并探讨了将其用作数据生成器的潜力。实证结果表明,我们数据集的规模与多样性可为语义解析研究者提供独特机遇。