Requirements engineering is a knowledge intensive process and crucial for the success of engineering projects. The field of knowledge-based requirements engineering (KBRE) aims to support engineers by providing knowledge to assist in the elicitation, validation, and management of system requirements. The advent of large language models (LLMs) opens new opportunities in the field of KBRE. This work experimentally investigates the potential of LLMs in requirements verification. Therein, LLMs are provided with a set of requirements and a textual system specification and are prompted to assess which requirements are fulfilled by the system specification. Different experimental variables such as system specification complexity, the number of requirements, and prompting strategies were analyzed. Formal rule-based systems serve as a benchmark to compare LLM performance to. Requirements and system specifications are derived from the smart-grid domain. Results show that advanced LLMs, like GPT-4o and Claude 3.5 Sonnet, achieved f1-scores between 79 % and 94 % in identifying non-fulfilled requirements, indicating potential for LLMs to be leveraged for requirements verification.

翻译：需求工程是一个知识密集型过程，对工程项目的成功至关重要。基于知识的需求工程领域旨在通过提供知识来支持工程师进行系统需求的获取、验证和管理。大语言模型的出现为基于知识的需求工程领域带来了新的机遇。本研究通过实验探讨了大语言模型在需求验证中的潜力。实验中，我们向大语言模型提供一组需求和一个文本形式的系统规范，并提示其评估系统规范满足了哪些需求。我们分析了系统规范复杂度、需求数量以及提示策略等不同实验变量。研究以基于规则的正式系统作为基准，与大语言模型的性能进行比较。需求和系统规范均源自智能电网领域。结果表明，先进的大语言模型（如GPT-4o和Claude 3.5 Sonnet）在识别未满足需求方面取得了79%至94%的F1分数，这表明大语言模型在需求验证方面具有应用潜力。