Current mobile GUI agent benchmarks systematically fail to assess memory capabilities, with only 5.2-11.8% memory-related tasks and no cross-session learning evaluation. We introduce MemGUI-Bench, a comprehensive memory-centric benchmark with pass@k and staged LLM-as-judge evaluation. Our contributions include: (1) a systematic memory taxonomy analyzing 11 agents across 5 architectures; (2) 128 tasks across 26 applications where 89.8% challenge memory through cross-temporal and cross-spatial retention; (3) MemGUI-Eval, an automated pipeline with Progressive Scrutiny and 7 hierarchical metrics; and (4) RQ-driven assessment of 11 state-of-the-art agents. Our experiments reveal significant memory deficits across all evaluated systems, identify 5 distinct failure modes, and synthesize 5 actionable design implications. All resources including code, benchmark, and evaluation results will be \textbf{\textit{fully open-sourced and continuously maintained}} at https://lgy0404.github.io/MemGUI-Bench/.

翻译：当前的移动GUI智能体基准测试在评估记忆能力方面存在系统性不足，仅有5.2%-11.8%的任务涉及记忆能力，且缺乏跨会话学习评估。我们提出了MemGUI-Bench，这是一个以记忆为中心的综合性基准，采用pass@k评估和分阶段LLM-as-judge评估方法。我们的贡献包括：（1）系统化的记忆分类体系，分析了涵盖5种架构的11种智能体；（2）涵盖26个应用程序的128项任务，其中89.8%通过跨时间与跨空间信息保持来挑战记忆能力；（3）MemGUI-Eval自动化评估流程，包含渐进式审查机制和7个层次化评估指标；（4）对11种前沿智能体进行研究问题驱动的评估。实验结果表明，所有被评估系统均存在显著的记忆缺陷，我们识别出5种不同的失效模式，并归纳出5项可指导实践的设计启示。所有资源，包括代码、基准测试集及评估结果，将在https://lgy0404.github.io/MemGUI-Bench/ \textbf{\textit{完全开源并持续维护}}。