Machine learning (ML) methods offer a wide range of configurable hyperparameters that have a significant influence on their performance. While accuracy is a commonly used performance objective, in many settings, it is not sufficient. Optimizing the ML models with respect to multiple objectives such as accuracy, confidence, fairness, calibration, privacy, latency, and memory consumption is becoming crucial. To that end, hyperparameter optimization, the approach to systematically optimize the hyperparameters, which is already challenging for a single objective, is even more challenging for multiple objectives. In addition, the differences in objective scales, the failures, and the presence of outlier values in objectives make the problem even harder. We propose a multi-objective Bayesian optimization (MoBO) algorithm that addresses these problems through uniform objective normalization and randomized weights in scalarization. We increase the efficiency of our approach by imposing constraints on the objective to avoid exploring unnecessary configurations (e.g., insufficient accuracy). Finally, we leverage an approach to parallelize the MoBO which results in a 5x speed-up when using 16x more workers.

翻译：机器学习方法提供了大量可配置的超参数，这些参数对其性能有显著影响。虽然准确率是常用的性能优化目标，但在许多场景中这并不足够。针对多个目标（如准确率、置信度、公平性、校准性、隐私性、延迟和内存消耗）优化机器学习模型正变得至关重要。为此，超参数优化——系统性地优化超参数的方法——对单一目标已具挑战性，而对多目标则更为困难。此外，目标尺度的差异、失败情况以及目标中异常值的存在进一步加剧了问题难度。我们提出一种多目标贝叶斯优化算法，通过均匀目标归一化和标量化中的随机权重来解决这些问题。通过为目标施加约束以避免探索不必要的配置（例如，不满足准确率要求），我们提高了方法的效率。最后，我们利用一种并行化多目标贝叶斯优化的方法，在使用16倍工作节点时可实现5倍加速。