The growing number of AI-driven applications in the mobile devices has led to solutions that integrate deep learning models with the available edge-cloud resources; due to multiple benefits such as reduction in on-device energy consumption, improved latency, improved network usage, and certain privacy improvements, split learning, where deep learning models are split away from the mobile device and computed in a distributed manner, has become an extensively explored topic. Combined with compression-aware methods where learning adapts to compression of communicated data, the benefits of this approach have further improved and could serve as an alternative to established approaches like federated learning methods. In this work, we develop an adaptive compression-aware split learning method ('deprune') to improve and train deep learning models so that they are much more network-efficient (use less network resources and are faster), which would make them ideal to deploy in weaker devices with the help of edge-cloud resources. This method is also extended ('prune') to very quickly train deep learning models, through a transfer learning approach, that trades off little accuracy for much more network-efficient inference abilities. We show that the 'deprune' method can reduce network usage by 4x when compared with a split-learning approach (that does not use our method) without loss of accuracy, while also improving accuracy over compression-aware split-learning by 4 percent. Lastly, we show that the 'prune' method can reduce the training time for certain models by up to 6x without affecting the accuracy when compared against a compression-aware split-learning approach.

翻译：移动设备中AI驱动应用的增长催生了将深度学习模型与可用边缘-云资源集成的解决方案；由于具备降低设备能耗、改善延迟、优化网络使用以及一定隐私提升等多重优势，将深度学习模型从移动设备拆分并以分布式方式计算的拆分学习已成为广泛探索的课题。结合学习过程自适应调整通信数据压缩的压缩感知方法，该方法的优势进一步增强，并可作为联邦学习等既有方法的替代方案。本研究开发了一种自适应压缩感知的拆分学习方法（"deprune"），用于改进和训练深度学习模型，使其更具网络效率（占用更少网络资源且速度更快），从而借助边缘-云资源更适用于弱性能设备部署。我们进一步扩展该方法（"prune"），通过迁移学习策略实现深度学习模型的快速训练，以轻微精度损失换取显著的网络高效推理能力。实验表明，与未采用本方法的拆分学习相比，"deprune"方法可在不损失精度的情况下将网络使用量降低4倍，同时相较于压缩感知拆分学习提升4%的精度。最后，我们证明"prune"方法可将特定模型的训练时间缩短至6倍，且与压缩感知拆分学习方法相比不影响模型精度。