Unlike existing work in deep neural network (DNN) graphs optimization for inference performance, we explore DNN graph optimization for energy awareness and savings for power- and resource-constrained machine learning devices. We present a method that allows users to optimize energy consumption or balance between energy and inference performance for DNN graphs. This method efficiently searches through the space of equivalent graphs, and identifies a graph and the corresponding algorithms that incur the least cost in execution. We implement the method and evaluate it with multiple DNN models on a GPU-based machine. Results show that our method achieves significant energy savings, i.e., 24% with negligible performance impact.

翻译：与现有针对推理性能的深度神经网络（DNN）图优化研究不同，本文探索面向能耗感知与节能的DNN图优化方法，旨在服务于功率与资源受限的机器学习设备。我们提出一种方法，允许用户针对DNN图优化能耗，或在能耗与推理性能之间进行权衡。该方法能高效搜索等价图空间，并识别出执行成本最低的图结构及其对应算法。我们实现了该方法，并在基于GPU的机器上使用多个DNN模型进行了评估。结果表明，我们的方法能够实现显著的节能效果，即在性能影响可忽略不计的情况下，节能达到24%。