The recent breakthroughs in machine learning (ML) and deep learning (DL) have catalyzed the design and development of various intelligent systems over wide application domains. While most existing machine learning models require large memory and computing power, efforts have been made to deploy some models on resource-constrained devices as well. A majority of the early application systems focused on exploiting the inference capabilities of ML and DL models, where data captured from different mobile and embedded sensing components are processed through these models for application goals such as classification and segmentation. More recently, the concept of exploiting the mobile and embedded computing resources for ML/DL model training has gained attention, as such capabilities allow (i) the training of models via local data without the need to share data over wireless links, thus enabling privacy-preserving computation by design, (ii) model personalization and environment adaptation, and (ii) deployment of accurate models in remote and hardly accessible locations without stable internet connectivity. This work targets to summarize and analyze state-of-the-art systems research that allows such on-device model training capabilities and provide a survey of on-device training from a systems perspective.

翻译：近期机器学习和深度学习的突破性进展推动了各类智能系统在广泛领域中的设计与开发。尽管现有大多数机器学习模型需要大量内存和计算能力，但研究人员已努力将部分模型部署在资源受限设备上。早期应用系统主要侧重于利用机器学习和深度学习模型的推理能力，即通过移动设备及嵌入式传感组件采集的数据经模型处理后实现分类、分割等应用目标。近期，利用移动和嵌入式计算资源进行模型训练的概念逐渐受到关注，此类能力可实现：（i）通过本地数据进行模型训练，无需通过无线链路共享数据，从而在设计层面实现隐私保护计算；（ii）模型个性化与环境自适应；（iii）在无稳定互联网连接的偏远及难以到达区域部署高精度模型。本研究旨在总结与分析当前支持此类设备端训练能力的系统前沿研究，并从系统视角对设备端训练进行综述。