Tiny deep learning has attracted increasing attention driven by the substantial demand for deploying deep learning on numerous intelligent Internet-of-Things devices. However, it is still challenging to unleash tiny deep learning's full potential on both large-scale datasets and downstream tasks due to the under-fitting issues caused by the limited model capacity of tiny neural networks (TNNs). To this end, we propose a framework called NetBooster to empower tiny deep learning by augmenting the architectures of TNNs via an expansion-then-contraction strategy. Extensive experiments show that NetBooster consistently outperforms state-of-the-art tiny deep learning solutions.

翻译：微型深度学习因其在众多智能物联网设备上部署深度学习的巨大需求而日益受到关注。然而，由于微型神经网络（TNNs）模型容量有限导致的欠拟合问题，在大型数据集和下游任务上充分释放微型深度学习的潜力仍具挑战性。为此，我们提出一种名为NetBooster的框架，通过"先扩张后收缩"策略增强TNNs的架构，从而赋能微型深度学习。大量实验表明，NetBooster在性能上持续优于当前最先进的微型深度学习方案。