REST APIs are widely used in industry, in all different kinds of domains. An example is Volkswagen AG, a German automobile manufacturer. Established testing approaches for REST APIs are time consuming, and require expertise from professional test engineers. Due to its cost and importance, in the scientific literature several approaches have been proposed to automatically test REST APIs. The open-source, search-based fuzzer EvoMaster is one of such tools proposed in the academic literature. However, how academic prototypes can be integrated in industry and have real impact to software engineering practice requires more investigation. In this paper, we report on our experience in using EvoMaster at Volkswagen AG, as an EvoMaster user from 2023 to 2026. We share our learnt lessons, and discuss several features needed to be implemented in EvoMaster to make its use in an industrial context successful. Feedback about value in industrial setups of EvoMaster was given from Volkswagen AG about 4 APIs. Additionally, a user study was conducted involving 11 testing specialists from 4 different companies. We further identify several real-world research challenges that still need to be solved.
翻译:REST API广泛应用于汽车制造等各类工业领域(例如德国大众汽车集团)。现有REST API测试方法耗时且依赖专业测试工程师的经验。鉴于其重要性与成本,学术界已提出多种自动化测试方案,其中基于搜索的开源模糊测试工具EvoMaster便是代表性学术原型之一。然而,如何将学术原型集成至工业环境并真正影响软件工程实践仍需深入探究。本文报告了作者作为EvoMaster用户(2023-2026年)在大众汽车集团的使用经验,分享了实践教训,并讨论为使该工具成功应用于工业环境所需实现的若干特性。大众汽车集团基于4个API对EvoMaster的工业价值进行了评估,同时我们对来自4家企业的11名测试专家开展了用户研究。此外,本文还揭示了一系列有待解决的现实研究挑战。