Scientific workflows are widely used to process large quantities of data, leading to significant energy consumption and carbon emissions. To reduce this environmental impact, energy and carbon-aware scheduling approaches could be employed. However, such methods require runtime and energy predictions, which are typically only available for workflows that have been executed previously. Meanwhile, scientists may execute new or modified workflows, use workflows with different input data, or run them on alternative infrastructure. To address this critical gap, we propose Augur, a novel method to predict the energy consumption of scientific workflow tasks prior to execution. By efficiently profiling both the available cluster infrastructure and the workflow at hand, Augur is capable of predicting the overall energy consumption of the workflow with a median prediction error of $16.3\pm15.3\%$ compared to Ichnos, an energy estimation method that uses fitted power models, and $18.2\pm14.7\%$ compared to Intel RAPL, as observed in our experimental evaluation on public and private cloud infrastructure. Relying on only minimal historical execution data, Augur outperforms two state-of-the-art methods in predicting both task runtime and total workflow energy, providing a robust foundation for energy-efficient and carbon-aware scientific data analysis.
翻译:科学工作流被广泛用于处理大量数据,导致巨大的能耗与碳排放。为减少这一环境影响,可采用面向能耗与碳感知的调度方法。然而,此类方法需要运行时和能耗预测,但通常仅对先前已执行的工作流可用。与此同时,科学家可能执行新的或修改过的工作流,或使用不同输入数据的工作流,或将其运行于替代基础设施之上。为填补这一关键空白,我们提出Augur,一种在执行前预测科学工作流任务能耗的新方法。通过高效分析可用集群基础设施与当前工作流,Augur能够预测工作流总能耗,其中位预测误差与使用拟合功率模型的能耗估计方法Ichnos相比为$16.3\pm15.3\%$,与Intel RAPL相比为$18.2\pm14.7\%$(如我们在公共与私有云基础设施上的实验评估所示)。仅依赖极少量历史执行数据,Augur在预测任务运行时与工作流总能耗方面优于两种最先进方法,为能耗高效且碳感知的科学数据分析提供了稳健基础。