Developing intelligent agents capable of operating a wide range of Graphical User Interfaces (GUIs) with human-level proficiency is a key milestone on the path toward Artificial General Intelligence. While most existing datasets and benchmarks for training and evaluating GUI agents are static and idealized, failing to reflect the complexity and unpredictability of real-world environments, particularly the presence of anomalies. To bridge this research gap, we propose D-GARA, a dynamic benchmarking framework, to evaluate Android GUI agent robustness in real-world anomalies. D-GARA introduces a diverse set of real-world anomalies that GUI agents commonly face in practice, including interruptions such as permission dialogs, battery warnings, and update prompts. Based on D-GARA framework, we construct and annotate a benchmark featuring commonly used Android applications with embedded anomalies to support broader community research. Comprehensive experiments and results demonstrate substantial performance degradation in state-of-the-art GUI agents when exposed to anomaly-rich environments, highlighting the need for robustness-aware learning. D-GARA is modular and extensible, supporting the seamless integration of new tasks, anomaly types, and interaction scenarios to meet specific evaluation goals.

翻译：开发能够以人类水平熟练操作各类图形用户界面（GUI）的智能体，是实现通用人工智能的关键里程碑。然而，当前用于训练和评估GUI智能体的大多数数据集与基准测试均为静态且理想化的，未能反映真实世界环境的复杂性与不可预测性，尤其缺乏对异常情况的考量。为填补这一研究空白，我们提出D-GARA——一个动态基准测试框架，用于评估Android GUI智能体在真实世界异常场景中的鲁棒性。D-GARA引入了一系列实践中GUI智能体常面临的真实异常类型，包括权限弹窗、电池警告、更新提示等中断事件。基于D-GARA框架，我们构建并标注了一个包含常用Android应用及嵌入异常场景的基准数据集，以支持更广泛的学术研究。综合实验结果表明，当前最先进的GUI智能体在异常密集环境中的性能显著下降，凸显了鲁棒性导向学习的必要性。D-GARA采用模块化可扩展设计，支持无缝集成新任务、异常类型与交互场景，以满足特定评估目标。