Hyperparameter optimization plays a pivotal role in enhancing the predictive performance and generalization capabilities of ML models. However, in many applications, we do not only care about predictive performance but also about objectives such as inference time, memory, or energy consumption. In such MOO scenarios, determining the importance of hyperparameters poses a significant challenge due to the complex interplay between the conflicting objectives. In this paper, we propose the first method for assessing the importance of hyperparameters in the context of multi-objective hyperparameter optimization. Our approach leverages surrogate-based hyperparameter importance (HPI) measures, i.e. fANOVA and ablation paths, to provide insights into the impact of hyperparameters on the optimization objectives. Specifically, we compute the a-priori scalarization of the objectives and determine the importance of the hyperparameters for different objective tradeoffs. Through extensive empirical evaluations on diverse benchmark datasets with three different objectives paired with accuracy, namely time, demographic parity, and energy consumption, we demonstrate the effectiveness and robustness of our proposed method. Our findings not only offer valuable guidance for hyperparameter tuning in MOO tasks but also contribute to advancing the understanding of HPI in complex optimization scenarios.

翻译：超参数优化在提升机器学习模型的预测性能与泛化能力方面扮演着关键角色。然而，在许多实际应用中，我们不仅关注预测性能，还同时关注推理时间、内存占用或能耗等目标。在这种多目标优化（MOO）场景下，由于相互冲突的目标之间存在复杂的交互作用，确定超参数的重要性成为一个重大挑战。本文首次提出了一种面向多目标超参数优化的超参数重要性评估方法。该方法利用基于代理模型的超参数重要性（HPI）度量标准（即fANOVA和消融路径），来揭示超参数对各优化目标的影响规律。具体而言，我们通过先验标量化方法处理目标函数，并针对不同目标权衡场景确定超参数的重要性。在包含时间、人口统计均等性和能耗三种不同目标与准确率组合的多样化基准数据集上进行的广泛实证评估，验证了所提方法的有效性与鲁棒性。本研究不仅为多目标优化任务中的超参数调优提供了有价值的指导，也推动了复杂优化场景下超参数重要性分析研究的深入发展。