The enormous energy consumption of machine learning (ML) and generative AI workloads shows no sign of waning, taking a toll on operating costs, power delivery, and environmental sustainability. Despite a long line of research on energy-efficient hardware, we found that software plays a critical role in ML energy optimization through two recent works: Zeus and Perseus. This is especially true for large language models (LLMs) because their model sizes and, therefore, energy demands are growing faster than hardware efficiency improvements. Therefore, we advocate for a cross-layer approach for energy optimizations in ML systems, where hardware provides architectural support that pushes energy-efficient software further, while software leverages and abstracts the hardware to develop techniques that bring hardware-agnostic energy-efficiency gains.

翻译：机器学习（ML）和生成式AI工作负载的巨大能耗至今未有减弱迹象，这给运营成本、电力传输和环境可持续性带来了沉重负担。尽管在节能硬件领域已有大量研究，但我们通过两项近期工作Zeus和Perseus发现，软件在机器学习能耗优化中发挥着关键作用。这对大语言模型（LLMs）尤为突出，因为其模型规模及相应能耗的增长速度已超过硬件效率的提升。因此，我们倡导在机器学习系统中采用跨层方法进行能耗优化：硬件提供架构支持以进一步推动节能软件的发展，而软件则利用并抽象化硬件特性，开发出能够实现硬件无关的能效提升技术。