Natural Language Processing (NLP) for Requirements Engineering (RE) (NLP4RE) seeks to apply NLP tools, techniques, and resources to the RE process to increase the quality of the requirements. There is little research involving the utilization of Generative AI-based NLP tools and techniques for requirements elicitation. In recent times, Large Language Models (LLM) like ChatGPT have gained significant recognition due to their notably improved performance in NLP tasks. To explore the potential of ChatGPT to assist in requirements elicitation processes, we formulated six questions to elicit requirements using ChatGPT. Using the same six questions, we conducted interview-based surveys with five RE experts from academia and industry and collected 30 responses containing requirements. The quality of these 36 responses (human-formulated + ChatGPT-generated) was evaluated over seven different requirements quality attributes by another five RE experts through a second round of interview-based surveys. In comparing the quality of requirements generated by ChatGPT with those formulated by human experts, we found that ChatGPT-generated requirements are highly Abstract, Atomic, Consistent, Correct, and Understandable. Based on these results, we present the most pressing issues related to LLMs and what future research should focus on to leverage the emergent behaviour of LLMs more effectively in natural language-based RE activities.
翻译:需求工程(RE)中的自然语言处理(NLP4RE)旨在将NLP工具、技术与资源应用于RE过程,以提升需求质量。目前,利用基于生成式AI的NLP工具和技术进行需求启发的研究尚不充分。近年来,以ChatGPT为代表的大语言模型(LLM)因在NLP任务中展现出显著提升的性能而备受关注。为探究ChatGPT在需求启发过程中的辅助潜力,我们设计了六个问题,并借助ChatGPT生成需求响应。随后,我们基于相同六个问题,对来自学术界和工业界的五位RE专家开展访谈式问卷调查,收集到30条包含需求的人类响应。通过第二轮访谈式问卷调查,另五位RE专家对这36条响应(人类制定+ChatGPT生成)的七项需求质量属性进行评估。通过比较ChatGPT生成需求与人类专家制定需求的质量,我们发现ChatGPT生成的需求在抽象性、原子性、一致性、正确性和可理解性方面表现优异。基于这些结果,我们提出当前LLM应用中最紧迫的问题,并指出未来研究应聚焦于更有效利用LLM涌现行为以支持基于自然语言的RE活动。