Automated Graphical User Interface (GUI) testing plays a crucial role in ensuring app quality, especially as mobile applications have become an integral part of our daily lives. Despite the growing popularity of learning-based techniques in automated GUI testing due to their ability to generate human-like interactions, they still suffer from several limitations, such as low testing coverage, inadequate generalization capabilities, and heavy reliance on training data. Inspired by the success of Large Language Models (LLMs) like ChatGPT in natural language understanding and question answering, we formulate the mobile GUI testing problem as a Q&A task. We propose GPTDroid, asking LLM to chat with the mobile apps by passing the GUI page information to LLM to elicit testing scripts, and executing them to keep passing the app feedback to LLM, iterating the whole process. Within this framework, we have also introduced a functionality-aware memory prompting mechanism that equips the LLM with the ability to retain testing knowledge of the whole process and conduct long-term, functionality-based reasoning to guide exploration. We evaluate it on 93 apps from Google Play and demonstrate that it outperforms the best baseline by 32% in activity coverage, and detects 31% more bugs at a faster rate. Moreover, GPTDroid identify 53 new bugs on Google Play, of which 35 have been confirmed and fixed.

翻译：自动化图形用户界面（GUI）测试在确保应用质量中扮演关键角色，尤其在移动应用已成为日常生活不可或缺组成部分的当下。尽管基于学习的技术因能生成类人交互而在自动化GUI测试中日益流行，但其仍存在测试覆盖率低、泛化能力不足、过度依赖训练数据等局限。受ChatGPT等大型语言模型（LLM）在自然语言理解与问答任务中取得成功的启发，我们将移动GUI测试问题形式化为问答任务，提出GPTDroid框架：通过向LLM传递GUI页面信息以生成测试脚本，执行脚本后持续将应用反馈返回LLM，循环迭代整个流程。在该框架中，我们进一步引入功能感知记忆提示机制，使LLM具备保留全流程测试知识的能力，并基于功能导向进行长期推理以指导探索。我们在Google Play的93款应用上开展评估，结果表明GPTDroid在活动覆盖率上超越最优基线方法32%，并以更快速度多检测31%的缺陷。此外，GPTDroid在Google Play上发现53个新缺陷，其中35个已获确认并修复。