The advent of large language models (LLMs) has opened up new opportunities in the field of mobile task automation. Their superior language understanding and reasoning capabilities allow users to automate complex and repetitive tasks. However, due to the inherent unreliability and high operational cost of LLMs, their practical applicability is quite limited. To address these issues, this paper introduces MobileGPT, an innovative LLM-based mobile task automator equipped with a human-like app memory. MobileGPT emulates the cognitive process of humans interacting with a mobile app -- explore, select, derive, and recall. This approach allows for a more precise and efficient learning of a task's procedure by breaking it down into smaller, modular sub-tasks that can be re-used, re-arranged, and adapted for various objectives. We implement MobileGPT using online LLMs services (GPT-3.5 and GPT-4) and evaluate its performance on a dataset of 160 user instructions across 8 widely used mobile apps. The results indicate that MobileGPT can automate and learn new tasks with 82.5% accuracy, and is able to adapt them to different contexts with near perfect (98.75%) accuracy while reducing both latency and cost by 62.5% and 68.8%, respectively, compared to the GPT-4 powered baseline.

翻译：大语言模型（LLM）的出现为移动任务自动化领域带来了新的机遇。其卓越的语言理解与推理能力使用户能够实现复杂且重复性任务的自动化。然而，由于LLM固有的不可靠性和高运行成本，其实际应用受到较大限制。为解决这些问题，本文提出MobileGPT——一种配备类人应用记忆的创新LLM移动任务自动化系统。MobileGPT模拟了人类与移动应用交互的认知过程——探索、选择、推导与回忆。该方法通过将任务过程分解为可复用、可重排、可适配不同目标的模块化子任务，实现了更精确高效的任务流程学习。我们利用在线LLM服务（GPT-3.5和GPT-4）实现MobileGPT，并在涵盖8款主流移动应用的160条用户指令数据集上评估其性能。结果表明，MobileGPT能以82.5%的准确率自动化并学习新任务，且能以近乎完美（98.75%）的准确率将其适配至不同场景，同时与基于GPT-4的基线相比，延迟和成本分别降低62.5%和68.8%。