The growing number of AI-driven applications in the mobile devices has led to solutions that integrate deep learning models with the available edge-cloud resources; due to multiple benefits such as reduction in on-device energy consumption, improved latency, improved network usage, and certain privacy improvements, split learning, where deep learning models are split away from the mobile device and computed in a distributed manner, has become an extensively explored topic. Combined with compression-aware methods where learning adapts to compression of communicated data, the benefits of this approach have further improved and could serve as an alternative to established approaches like federated learning methods. In this work, we develop an adaptive compression-aware split learning method ('deprune') to improve and train deep learning models so that they are much more network-efficient (use less network resources and are faster), which would make them ideal to deploy in weaker devices with the help of edge-cloud resources. This method is also extended ('prune') to very quickly train deep learning models, through a transfer learning approach, that trades off little accuracy for much more network-efficient inference abilities. We show that the 'deprune' method can reduce network usage by 4x when compared with a split-learning approach (that does not use our method) without loss of accuracy, while also improving accuracy over compression-aware split-learning by 4 percent. Lastly, we show that the 'prune' method can reduce the training time for certain models by up to 6x without affecting the accuracy when compared against a compression-aware split-learning approach.

翻译：移动设备中人工智能驱动应用数量的增长，催生了将深度学习模型与可用边缘-云资源相结合的解决方案。由于具备降低设备能耗、改善延迟、优化网络使用以及提升隐私保护等多重优势，将深度学习模型从移动设备拆分并以分布式方式计算的拆分学习已成为广泛探索的课题。当与学习适应通信数据压缩的压缩感知方法相结合时，该方法的优势得到进一步增强，并可作为联邦学习等现有方法的替代方案。本研究提出一种自适应压缩感知的拆分学习方法（'deprune'），用于改进和训练深度学习模型，使其更具网络效率（减少网络资源消耗并提升速度），从而使模型能够在边缘-云资源的辅助下部署于性能较弱的设备。该方法还扩展出一种快速训练深度学习模型的方式（'prune'），通过迁移学习以少量精度损失换取更强的网络高效推理能力。实验表明，与未采用本方法的拆分学习相比，'deprune'方法在不损失精度的前提下将网络资源消耗降低4倍，同时相比压缩感知拆分学习可提升4%的精度。最后，我们证明与压缩感知拆分学习相比，'prune'方法可将特定模型的训练时间缩短至1/6，且不影响精度。