Batteryless systems frequently face power failures, requiring extra runtime buffers to maintain inference progress and leaving only a memory space for storing ultra-tiny deep neural networks (DNNs). Besides, making these models responsive to stochastic energy harvesting dynamics during inference requires a balance between inference accuracy, latency, and energy overhead. Recent works on compression mostly focus on time and memory, but often ignore energy dynamics or significantly reduce the accuracy of pre-trained DNNs. Existing energy-adaptive inference works modify the architecture of pre-trained models and have significant memory overhead. Thus, energy-adaptive and accurate inference of pre-trained DNNs on batteryless devices with extreme memory constraints is more challenging than traditional microcontrollers. We combat these issues by proposing FreeML, a framework to optimize pre-trained DNN models for memory-efficient and energy-adaptive inference on batteryless systems. FreeML comprises (1) a novel compression technique to reduce the model footprint and runtime memory requirements simultaneously, making them executable on extremely memory-constrained batteryless platforms; and (2) the first early exit mechanism that uses a single exit branch for all exit points to terminate inference at any time, making models energy-adaptive with minimal memory overhead. Our experiments showed that FreeML reduces the model sizes by up to $95 \times$, supports adaptive inference with a $2.03-19.65 \times$ less memory overhead, and provides significant time and energy benefits with only a negligible accuracy drop compared to the state-of-the-art.

翻译：无电池系统频繁遭遇电力中断，需额外运行时缓冲区维持推理进程，仅剩极小小型深度神经网络（DNN）的存储空间。此外，在推理过程中使模型对随机能量收集动态做出响应，需在推理精度、延迟与能量开销间寻求平衡。近期压缩研究工作主要关注时间与内存，但常忽略能量动态或显著降低预训练DNN的精度。现有能量自适应推理工作会修改预训练模型的架构，并产生显著的内存开销。因此，在内存极度受限的无电池设备上实现预训练DNN的能量自适应且高精度推理，比传统微控制器更具挑战性。为解决这些问题，我们提出FreeML框架，该框架优化预训练DNN模型，使其在无电池系统上支持内存高效与能量自适应推理。FreeML包含：（1）一种新型压缩技术，可同时减小模型足迹与运行时内存需求，使其能在内存极度受限的无电池平台上执行；（2）首个使用单退出分支覆盖所有退出点、可随时终止推理的早期退出机制，以最小内存开销实现模型能量自适应。实验表明，与现有最优方法相比，FreeML可将模型体积缩减高达$95 \times$，以$2.03-19.65 \times$更低内存开销支持自适应推理，并在仅产生可忽略精度损失的前提下提供显著的时间与能量效益。