Recent breakthrough technological progressions of powerful mobile computing resources such as low-cost mobile GPUs along with cutting-edge, open-source software architectures have enabled high-performance deep learning on mobile platforms. These advancements have revolutionized the capabilities of today's mobile applications in different dimensions to perform data-driven intelligence locally, particularly for smart health applications. Unlike traditional machine learning (ML) architectures, modern on-device deep learning frameworks are proficient in utilizing computing resources in mobile platforms seamlessly, in terms of producing highly accurate results in less inference time. However, on the flip side, energy resources in a mobile device are typically limited. Hence, whenever a complex Deep Neural Network (DNN) architecture is fed into the on-device deep learning framework, while it achieves high prediction accuracy (and performance), it also urges huge energy demands during the runtime. Therefore, managing these resources efficiently within the spectrum of performance and energy efficiency is the newest challenge for any mobile application featuring data-driven intelligence beyond experimental evaluations. In this paper, first, we provide a timely review of recent advancements in on-device deep learning while empirically evaluating the performance metrics of current state-of-the-art ML architectures and conventional ML approaches with the emphasis given on energy characteristics by deploying them on a smart health application. With that, we are introducing a new framework through an energy-aware, adaptive model comprehension and realization (EAMCR) approach that can be utilized to make more robust and efficient inference decisions based on the available computing/energy resources in the mobile device during the runtime.

翻译：近期，低成本移动GPU等强大移动计算资源的技术突破，以及前沿开源软件架构的进步，使得在移动平台上实现高性能深度学习成为可能。这些进展从不同维度彻底革新了当今移动应用在本地执行数据驱动智能的能力，尤其是在智能健康应用领域。与传统机器学习架构不同，现代设备端深度学习框架能够无缝利用移动平台的计算资源，在更短的推理时间内产出高精度结果。然而，另一方面，移动设备中的能耗资源通常有限。因此，一旦将复杂的深度神经网络架构部署至设备端深度学习框架，虽然能实现高预测精度（与性能），但也将在运行时产生巨大的能耗需求。因此，如何在性能与能效的权衡中高效管理这些资源，已成为超越实验评估的、任何具备数据驱动智能的移动应用面临的最新挑战。本文首先及时综述了设备端深度学习的最新进展，同时通过将当前最先进的机器学习架构与传统机器学习方法部署于智能健康应用，重点基于能耗特性对它们的性能指标进行了实证评估。在此基础上，我们提出了一种新的框架——基于能耗感知的自适应模型理解与实现方法，该方法可根据移动设备运行时的可用计算/能耗资源，做出更鲁棒且高效的推理决策。