Tests executed by human testers are still widespread in practice and fill the gap left by limitations of automated approaches. Among the human-centered approaches, exploratory testing is the de facto approach in agile teams. Although it is focused on the expertise and creativity of the tester, the activity of exploratory testing may benefit from support provided by an automated agent that interacts with the human testers. This paper presents a chatbot, called BotExpTest, designed to support testers while performing exploratory tests of software applications. We implemented BotExpTest on top of the instant messaging social platform Discord; this version includes functionalities to report bugs and issues, time management of test sessions, guidelines for app testing, and presentation of exploratory testing strategies. To assess BotExpTest, we conducted a user study with six software engineering professionals. They carried out two sessions performing exploratory tests along with BotExpTest. Participants were capable of revealing bugs and found the experience to interact with the chatbot positive. Preliminary analyses indicate that chatbot-enabled exploratory testing may be as effective as similar approaches and help testers to uncover different bugs. Bots are shown to be valuable resources for Software Engineering, and initiatives like BotExpTest may help to improve the effectiveness of testing activities like exploratory testing.
翻译:人工测试人员的测试在实践中仍广泛存在,填补了自动化方法局限性的空白。在以人为中心的测试方法中,探索性测试是敏捷团队的事实标准方法。尽管探索性测试依赖于测试人员的专业知识和创造力,但该活动可能受益于与人工测试人员交互的自动化代理所提供的支持。本文介绍了一款名为BotExpTest的聊天机器人,旨在支持测试人员执行软件应用的探索性测试。我们在即时通讯社交平台Discord之上实现了BotExpTest;该版本功能包括报告故障与问题、测试会话时间管理、应用测试指南以及探索性测试策略展示。为评估BotExpTest,我们开展了一项包含六名软件工程专业人员的用户研究。他们进行了两轮使用BotExpTest执行探索性测试的会话。参与者能够发现故障,并对与聊天机器人的交互体验给予正面评价。初步分析表明,基于聊天机器人的探索性测试在有效性方面可与类似方法相当,并有助于测试人员发现不同类型的故障。研究显示,机器人为软件工程提供了宝贵资源,而BotExpTest等举措可能有助于提升探索性测试等测试活动的效果。