The Object Constraint Language (OCL) is a declarative language that adds constraints and object query expressions to MOF models. Despite its potential to provide precision and conciseness to UML models, the unfamiliar syntax of OCL has hindered its adoption. Recent advancements in LLMs, such as GPT-3, have shown their capability in many NLP tasks, including semantic parsing and text generation. Codex, a GPT-3 descendant, has been fine-tuned on publicly available code from GitHub and can generate code in many programming languages. We investigate the reliability of OCL constraints generated by Codex from natural language specifications. To achieve this, we compiled a dataset of 15 UML models and 168 specifications and crafted a prompt template with slots to populate with UML information and the target task, using both zero- and few-shot learning methods. By measuring the syntactic validity and execution accuracy metrics of the generated OCL constraints, we found that enriching the prompts with UML information and enabling few-shot learning increases the reliability of the generated OCL constraints. Furthermore, the results reveal a close similarity based on sentence embedding between the generated OCL constraints and the human-written ones in the ground truth, implying a level of clarity and understandability in the generated OCL constraints by Codex.

翻译：对象约束语言（OCL）是一种声明性语言，可为MOF模型添加约束和对象查询表达式。尽管OCL有潜力为UML模型提供精确性和简洁性，但其不熟悉的语法阻碍了其应用。近年来，大型语言模型（如GPT-3）在诸多自然语言处理任务（包括语义解析和文本生成）中展现出强大能力。作为GPT-3的衍生模型，Codex基于GitHub上的公开代码进行了微调，能够生成多种编程语言的代码。我们研究了Codex根据自然语言规范生成的OCL约束的可靠性。为此，我们整理了一个包含15个UML模型和168条规范的数据集，并设计了一个带有可填充UML信息及目标任务占位符的提示模板，同时采用零样本和少样本学习方法。通过测量生成的OCL约束的语法有效性和执行准确性指标，我们发现：向提示中丰富UML信息并结合少样本学习，能提高生成OCL约束的可靠性。此外，结果表明，生成的OCL约束与真实人工编写的约束在句嵌入层面高度相似，这暗示了Codex生成的OCL约束具备一定的清晰度和可理解性。