Tiny Machine Learning (TinyML) is a new frontier of machine learning. By squeezing deep learning models into billions of IoT devices and microcontrollers (MCUs), we expand the scope of AI applications and enable ubiquitous intelligence. However, TinyML is challenging due to hardware constraints: the tiny memory resource makes it difficult to hold deep learning models designed for cloud and mobile platforms. There is also limited compiler and inference engine support for bare-metal devices. Therefore, we need to co-design the algorithm and system stack to enable TinyML. In this review, we will first discuss the definition, challenges, and applications of TinyML. We then survey the recent progress in TinyML and deep learning on MCUs. Next, we will introduce MCUNet, showing how we can achieve ImageNet-scale AI applications on IoT devices with system-algorithm co-design. We will further extend the solution from inference to training and introduce tiny on-device training techniques. Finally, we present future directions in this area. Today's large model might be tomorrow's tiny model. The scope of TinyML should evolve and adapt over time.

翻译：微型机器学习（TinyML）是机器学习的新前沿。通过将深度学习模型压缩到数十亿的物联网设备和微控制器中，我们扩展了人工智能应用的范围，并实现了普适智能。然而，由于硬件限制，TinyML面临诸多挑战：有限的存储资源使得针对云端和移动平台设计的深度学习模型难以部署，同时裸机设备上编译器与推理引擎的支持也十分有限。因此，我们需要对算法和系统栈进行协同设计，以实现TinyML。本综述首先讨论TinyML的定义、挑战和应用，随后综述该领域及MCU上深度学习的最新进展。接着，我们将介绍MCUNet，展示如何通过系统-算法协同设计在物联网设备上实现ImageNet规模的人工智能应用。我们还将从推理扩展到训练，介绍微型设备上的训练技术。最后，我们展望该领域的未来方向——今日的大模型或许将成为明日的小模型，TinyML的范畴也应随时代演进与调整。