Today, deep learning optimization is primarily driven by research focused on achieving high inference accuracy and reducing latency. However, the energy efficiency aspect is often overlooked, possibly due to a lack of sustainability mindset in the field and the absence of a holistic energy dataset. In this paper, we conduct a threefold study, including energy measurement, prediction, and efficiency scoring, with an objective to foster transparency in power and energy consumption within deep learning across various edge devices. Firstly, we present a detailed, first-of-its-kind measurement study that uncovers the energy consumption characteristics of on-device deep learning. This study results in the creation of three extensive energy datasets for edge devices, covering a wide range of kernels, state-of-the-art DNN models, and popular AI applications. Secondly, we design and implement the first kernel-level energy predictors for edge devices based on our kernel-level energy dataset. Evaluation results demonstrate the ability of our predictors to provide consistent and accurate energy estimations on unseen DNN models. Lastly, we introduce two scoring metrics, PCS and IECS, developed to convert complex power and energy consumption data of an edge device into an easily understandable manner for edge device end-users. We hope our work can help shift the mindset of both end-users and the research community towards sustainability in edge computing, a principle that drives our research. Find data, code, and more up-to-date information at https://amai-gsu.github.io/DeepEn2023.

翻译：当前，深度学习优化研究主要聚焦于实现高推理精度和降低延迟。然而，能效方面常被忽视，这可能是由于该领域缺乏可持续性思维以及缺乏全面的能源数据集。本文开展了一项包含能源测量、预测和效率评分的三方面研究，旨在促进不同边缘设备上深度学习功耗与能耗的透明度。首先，我们提出了一项详细且首创的测量研究，揭示了设备端深度学习的能耗特性。该研究创建了三个针对边缘设备的广泛能源数据集，涵盖了广泛的算子内核、最先进的DNN模型和流行的AI应用。其次，基于我们的内核级能源数据集，我们设计并实现了首个面向边缘设备的内核级能耗预测器。评估结果表明，我们的预测器能够在未见过的DNN模型上提供一致且准确的能耗估计。最后，我们引入了两个评分指标——PCS和IECS，旨在将边缘设备复杂的功耗和能耗数据转换为易于边缘设备终端用户理解的形式。我们希望我们的工作能够帮助终端用户和研究社区将思维转向边缘计算的可持续性，这一原则驱动着我们的研究。数据、代码及最新信息请访问 https://amai-gsu.github.io/DeepEn2023。