Bayesian Optimization (BO) is a class of black-box, surrogate-based heuristics that can efficiently optimize problems that are expensive to evaluate, and hence admit only small evaluation budgets. BO is particularly popular for solving numerical optimization problems in industry, where the evaluation of objective functions often relies on time-consuming simulations or physical experiments. However, many industrial problems depend on a large number of parameters. This poses a challenge for BO algorithms, whose performance is often reported to suffer when the dimension grows beyond 15 variables. Although many new algorithms have been proposed to address this problem, it is not well understood which one is the best for which optimization scenario. In this work, we compare five state-of-the-art high-dimensional BO algorithms, with vanilla BO and CMA-ES on the 24 BBOB functions of the COCO environment at increasing dimensionality, ranging from 10 to 60 variables. Our results confirm the superiority of BO over CMA-ES for limited evaluation budgets and suggest that the most promising approach to improve BO is the use of trust regions. However, we also observe significant performance differences for different function landscapes and budget exploitation phases, indicating improvement potential, e.g., through hybridization of algorithmic components.

翻译：贝叶斯优化（BO）是一类基于黑箱代理模型的启发式算法，能够高效优化评估代价昂贵、因而仅允许少量评估预算的问题。BO在工业领域求解数值优化问题中尤为流行，该类问题中目标函数的评估通常依赖耗时仿真或物理实验。然而，许多工业问题依赖于大量参数，这对BO算法构成了挑战——文献普遍报告当维度超过15个变量时其性能会下降。尽管已有诸多新算法被提出以解决该问题，但何种算法在何种优化场景中最优仍不明确。本研究在COCO基准环境的24个BBOB函数上，对比了五种先进的高维BO算法、标准BO及CMA-ES（协方差矩阵自适应进化策略），测试维度范围从10个变量递增至60个变量。结果证实了在有限评估预算下BO优于CMA-ES，并表明改进BO最有前景的方法是引入信任区域（trust-regions）。然而，我们也观察到不同函数形态和预算利用阶段存在显著性能差异，表明通过算法组件混合（如杂交策略）仍具有改进潜力。