Graphical User Interfaces (GUIs) are the primary means by which users interact with mobile applications, making them crucial to both app functionality and user experience. However, a major challenge in automated testing is the frequent appearance of app-blocking pop-ups, such as ads or system alerts, which obscure critical UI elements and disrupt test execution, often requiring manual intervention. These interruptions lead to inaccurate test results, increased testing time, and reduced reliability, particularly for stakeholders conducting large-scale app testing. To address this issue, we introduce PopSweeper, a novel tool designed to detect and resolve app-blocking pop-ups in real-time during automated GUI testing. PopSweeper combines deep learning-based computer vision techniques for pop-up detection and close button localization, allowing it to autonomously identify pop-ups and ensure uninterrupted testing. We evaluated PopSweeper on over 72K app screenshots from the RICO dataset and 87 top-ranked mobile apps collected from app stores, manually identifying 832 app-blocking pop-ups. PopSweeper achieved 91.7% precision and 93.5% recall in pop-up classification and 93.9% BoxAP with 89.2% recall in close button detection. Furthermore, end-to-end evaluations demonstrated that PopSweeper successfully resolved blockages in 87.1% of apps with minimal overhead, achieving classification and close button detection within 60 milliseconds per frame. These results highlight PopSweeper's capability to enhance the accuracy and efficiency of automated GUI testing by mitigating pop-up interruptions.
翻译:图形用户界面(GUI)是用户与移动应用交互的主要方式,对应用功能和用户体验至关重要。然而,自动化测试面临的一个主要挑战是频繁出现的应用阻塞型弹窗(如广告或系统提示),这些弹窗会遮挡关键UI元素并中断测试执行,通常需要人工干预。此类中断导致测试结果不准确、测试时间增加以及可靠性降低,对于进行大规模应用测试的相关方尤为突出。为解决这一问题,我们提出了PopSweeper——一种在自动化GUI测试过程中实时检测与消除应用阻塞型弹窗的新型工具。PopSweeper融合了基于深度学习的计算机视觉技术,用于弹窗检测与关闭按钮定位,使其能够自主识别弹窗并确保测试流程不被中断。我们在来自RICO数据集的超过7.2万张应用截图以及从应用商店收集的87款热门移动应用上对PopSweeper进行了评估,人工标注了832个应用阻塞型弹窗。实验表明:PopSweeper在弹窗分类任务中实现了91.7%的精确率与93.5%的召回率;在关闭按钮检测任务中达到93.9%的BoxAP与89.2%的召回率。进一步的端到端评估显示,PopSweeper能以每帧60毫秒的处理速度完成分类与关闭按钮检测,在87.1%的应用中成功消除阻塞且开销极小。这些结果证明了PopSweeper通过有效抑制弹窗干扰,能够显著提升自动化GUI测试的准确性与执行效率。