In today's landscape, smartphones have evolved into hubs for hosting a multitude of deep learning models aimed at local execution. A key realization driving this work is the notable fragmentation among these models, characterized by varied architectures, operators, and implementations. This fragmentation imposes a significant burden on the comprehensive optimization of hardware, system settings, and algorithms. Buoyed by the recent strides in large foundation models, this work introduces a pioneering paradigm for mobile AI: a collaborative management approach between the mobile OS and hardware, overseeing a foundational model capable of serving a broad spectrum of mobile AI tasks, if not all. This foundational model resides within the NPU and remains impervious to app or OS revisions, akin to firmware. Concurrently, each app contributes a concise, offline fine-tuned "adapter" tailored to distinct downstream tasks. From this concept emerges a concrete instantiation known as \sys. It amalgamates a curated selection of publicly available Large Language Models (LLMs) and facilitates dynamic data flow. This concept's viability is substantiated through the creation of an exhaustive benchmark encompassing 38 mobile AI tasks spanning 50 datasets, including domains such as Computer Vision (CV), Natural Language Processing (NLP), audio, sensing, and multimodal inputs. Spanning this benchmark, \sys unveils its impressive performance. It attains accuracy parity in 85\% of tasks, demonstrates improved scalability in terms of storage and memory, and offers satisfactory inference speed on Commercial Off-The-Shelf (COTS) mobile devices fortified with NPU support. This stands in stark contrast to task-specific models tailored for individual applications.

翻译：在当今格局中，智能手机已演变为承载众多深度学习模型进行本地执行的枢纽。这项工作背后的关键洞察在于这些模型之间存在显著碎片化现象，表现为架构、算子及实现的多样性。这种碎片化给硬件、系统设置与算法的全面优化带来了沉重负担。受近期大基础模型进展的推动，本文提出了一种移动AI开创性范式：移动操作系统与硬件协同管理一种基础模型，使其能够服务于（若非全部）绝大多数移动AI任务。该基础模型驻留在NPU中，并像固件一样不受应用或操作系统版本更新的影响。同时，每个应用贡献一个针对特定下游任务进行轻量级离线微调的"适配器"。基于这一概念，我们实现了具体实例化系统\sys，该系统融合了精心挑选的公开大语言模型（LLM），并支持动态数据流。通过构建涵盖38项移动AI任务（涉及50个数据集，包含计算机视觉、自然语言处理、音频、传感及多模态输入等领域）的全面基准测试，验证了该概念的可行性。在该基准测试中，\sys展现了卓越性能：在85%的任务上达到精度持平，在存储和内存方面展现出更优的可扩展性，并在配备NPU支持的商用现成（COTS）移动设备上提供了令人满意的推理速度。这与针对单个应用定制的任务专用模型形成了鲜明对比。