As REST APIs become an increasingly significant part of software systems, their validation is becoming more critical. Hence, testing and uncovering underlying issues are of utmost importance for improving software quality. However, testing REST APIs is challenging mainly due to the difficulty of assessing whether the output of an API call is correct, i.e., the test oracle problem. Metamorphic testing is a specification-based testing approach for situations where correct outputs are unknown or not specified explicitly. To check the correctness of a system, relations between the different outputs are specified. We present ARMeta, a tool-supported approach that uses an LLM-based multi-agent workflow to support metamorphic testing of REST APIs documented with OpenAPI. The agentic workflow is used to identify metamorphic test scenarios and specify them in the Given-When-Then format. These scenarios are automatically implemented as executable tests and executed against the system under test. We evaluate ARMeta on two publicly available web applications that expose REST interfaces and compare its performance with a scenario-based testing baseline. The results show that ARMeta explores behaviors that serve as a complement to existing scenario-based testing approaches.

翻译：随着REST API在软件系统中日益重要，其验证工作也变得更加关键。因此，通过测试来发现潜在问题对于提升软件质量具有至关重要的意义。然而，REST API测试的主要挑战在于难以判断API调用的输出是否正确，即测试预言机问题。蜕变测试是一种基于规约的测试方法，适用于正确输出未知或未明确指定的情况。它通过定义不同输出之间的关系来检验系统正确性。本文提出ARMeta工具化方法，采用基于大语言模型的多智能体工作流，支持对通过OpenAPI文档描述的REST API进行蜕变测试。该智能体工作流用于识别蜕变测试场景，并以Given-When-Then格式进行规约描述。这些场景将自动实现为可执行测试用例，并在被测系统上执行。我们在两个公开的REST接口Web应用上对ARMeta进行评估，并将其性能与基于场景的测试基线进行对比。结果表明，ARMeta探索的行为能够有效补充现有基于场景的测试方法。