Expected Improvement (EI) is arguably the most popular acquisition function in Bayesian optimization and has found countless successful applications, but its performance is often exceeded by that of more recent methods. Notably, EI and its variants, including for the parallel and multi-objective settings, are challenging to optimize because their acquisition values vanish numerically in many regions. This difficulty generally increases as the number of observations, dimensionality of the search space, or the number of constraints grow, resulting in performance that is inconsistent across the literature and most often sub-optimal. Herein, we propose LogEI, a new family of acquisition functions whose members either have identical or approximately equal optima as their canonical counterparts, but are substantially easier to optimize numerically. We demonstrate that numerical pathologies manifest themselves in "classic" analytic EI, Expected Hypervolume Improvement (EHVI), as well as their constrained, noisy, and parallel variants, and propose corresponding reformulations that remedy these pathologies. Our empirical results show that members of the LogEI family of acquisition functions substantially improve on the optimization performance of their canonical counterparts and surprisingly, are on par with or exceed the performance of recent state-of-the-art acquisition functions, highlighting the understated role of numerical optimization in the literature.

翻译：期望改进（EI）无疑是贝叶斯优化中最流行的采集函数，已取得无数成功应用，但其性能常被更近期的方法超越。值得注意的是，EI及其变体（包括并行和多目标设置）的优化颇具挑战性，因为其采集值在许多区域数值上趋于零。这一困难通常随着观测次数、搜索空间维度或约束数量的增加而加剧，导致文献中性能表现不一致且大多次优。本文提出LogEI这一新型采集函数族，其成员与标准对应函数具有相同或近似相等的最优解，但数值优化难度显著降低。我们证明了数值病态现象存在于"经典"解析型EI、期望超体积改进（EHVI）及其带约束、含噪声和并行变体中，并提出相应的修正公式以消除这些病态。实验结果表明，LogEI采集函数族的成员在优化性能上显著优于其标准对应函数，且出人意料地与近期最先进采集函数性能持平或更优，这凸显了数值优化在文献中常被低估的作用。