The importance of computational modeling of mobile user interfaces (UIs) is undeniable. However, these require a high-quality UI dataset. Existing datasets are often outdated, collected years ago, and are frequently noisy with mismatches in their visual representation. This presents challenges in modeling UI understanding in the wild. This paper introduces a novel approach to automatically mine UI data from Android apps, leveraging Large Language Models (LLMs) to mimic human-like exploration. To ensure dataset quality, we employ the best practices in UI noise filtering and incorporate human annotation as a final validation step. Our results demonstrate the effectiveness of LLMs-enhanced app exploration in mining more meaningful UIs, resulting in a large dataset MUD of 18k human-annotated UIs from 3.3k apps. We highlight the usefulness of MUD in two common UI modeling tasks: element detection and UI retrieval, showcasing its potential to establish a foundation for future research into high-quality, modern UIs.

翻译：移动用户界面（UI）的计算建模重要性毋庸置疑。然而，这需要高质量的UI数据集。现有数据集往往过时（收集于数年前），且常因可视化表示不匹配而存在噪声，给实际场景中的UI理解建模带来挑战。本文提出一种新颖方法，利用大语言模型（LLMs）模拟人类探索行为，从Android应用自动挖掘UI数据。为确保数据集质量，我们采用UI噪声过滤的最佳实践，并将人工标注作为最终验证环节。结果表明，LLMs增强型应用探索能有效挖掘更有意义的UI，最终构建了包含3.3k个应用中18k个人工标注UI的大规模数据集MUD。我们通过元素检测与UI检索两项常见UI建模任务验证了MUD的实用性，展示了其作为高质量现代UI研究基础数据集的潜力。