Bayesian optimization (BO) is an efficient framework for optimization of black-box objectives when function evaluations are costly and gradient information is not easily accessible. BO has been successfully applied to automate the task of hyperparameter optimization (HPO) in machine learning (ML) models with the primary objective of optimizing predictive performance on held-out data. In recent years, however, with ever-growing model sizes, the energy cost associated with model training has become an important factor for ML applications. Here we evaluate Constrained Bayesian Optimization (CBO) with the primary objective of minimizing energy consumption and subject to the constraint that the generalization performance is above some threshold. We evaluate our approach on regression and classification tasks and demonstrate that CBO achieves lower energy consumption without compromising the predictive performance of ML models.

翻译：贝叶斯优化（BO）是一种高效的框架，适用于目标函数评估成本高昂且梯度信息不易获取的黑箱目标优化问题。BO已成功应用于机器学习（ML）模型的超参数优化（HPO）自动化任务，其主要目标是优化模型在预留数据上的预测性能。然而近年来，随着模型规模的不断增长，模型训练相关的能耗成本已成为机器学习应用中的重要考量因素。本文评估了约束贝叶斯优化（CBO）方法，其主要目标是最小化能耗，并受限于泛化性能需高于特定阈值的约束条件。我们在回归和分类任务上评估了该方法，结果表明CBO能够在保证机器学习模型预测性能的前提下实现更低的能耗。