Today, deep learning optimization is primarily driven by research focused on achieving high inference accuracy and reducing latency. However, the energy efficiency aspect is often overlooked, possibly due to a lack of sustainability mindset in the field and the absence of a holistic energy dataset. In this paper, we conduct a threefold study, including energy measurement, prediction, and efficiency scoring, with an objective to foster transparency in power and energy consumption within deep learning across various edge devices. Firstly, we present a detailed, first-of-its-kind measurement study that uncovers the energy consumption characteristics of on-device deep learning. This study results in the creation of three extensive energy datasets for edge devices, covering a wide range of kernels, state-of-the-art DNN models, and popular AI applications. Secondly, we design and implement the first kernel-level energy predictors for edge devices based on our kernel-level energy dataset. Evaluation results demonstrate the ability of our predictors to provide consistent and accurate energy estimations on unseen DNN models. Lastly, we introduce two scoring metrics, PCS and IECS, developed to convert complex power and energy consumption data of an edge device into an easily understandable manner for edge device end-users. We hope our work can help shift the mindset of both end-users and the research community towards sustainability in edge computing, a principle that drives our research. Find data, code, and more up-to-date information at https://amai-gsu.github.io/DeepEn2023.

翻译：当前，深度学习优化主要围绕高推理精度与低延迟展开研究，然而能效层面常被忽视，其原因可能在于该领域缺乏可持续性思维，且缺乏整体性能耗数据集。本文从能量测量、预测与效率评分三个维度展开研究，旨在提升不同边缘设备上深度学习功耗与能耗的透明度。首先，我们首次提出一项详细的测量研究，揭示了设备端深度学习的能耗特征，并据此构建了三个覆盖广泛内核、先进深度神经网络模型及主流人工智能应用的边缘设备能耗数据集。其次，基于核级能耗数据集，我们设计并实现了首个面向边缘设备的核级能量预测器。评估结果表明，该预测器能够为未见过的深度神经网络模型提供一致且准确的能量估算。最后，我们提出两种评分指标——PCS（功耗一致性分数）和IECS（能效一致性分数），旨在将边缘设备的复杂功耗与能耗数据转化为终端用户易于理解的形式。我们期望这项工作能推动终端用户与研究社区转向边缘计算的可持续性理念——这一原则正是本研究的核心驱动力。数据、代码及更多最新信息请访问https://amai-gsu.github.io/DeepEn2023。