Expected Improvement (EI) is arguably the most popular acquisition function in Bayesian optimization and has found countless successful applications, but its performance is often exceeded by that of more recent methods. Notably, EI and its variants, including for the parallel and multi-objective settings, are challenging to optimize because their acquisition values vanish numerically in many regions. This difficulty generally increases as the number of observations, dimensionality of the search space, or the number of constraints grow, resulting in performance that is inconsistent across the literature and most often sub-optimal. Herein, we propose LogEI, a new family of acquisition functions whose members either have identical or approximately equal optima as their canonical counterparts, but are substantially easier to optimize numerically. We demonstrate that numerical pathologies manifest themselves in "classic" analytic EI, Expected Hypervolume Improvement (EHVI), as well as their constrained, noisy, and parallel variants, and propose corresponding reformulations that remedy these pathologies. Our empirical results show that members of the LogEI family of acquisition functions substantially improve on the optimization performance of their canonical counterparts and surprisingly, are on par with or exceed the performance of recent state-of-the-art acquisition functions, highlighting the understated role of numerical optimization in the literature.

翻译：期望改进（EI）可以说是贝叶斯优化中最常用的采集函数，并已在无数应用中取得成功，但其性能常常被更近期的方法所超越。值得注意的是，EI及其变体（包括用于并行和多目标设置的变体）在优化中面临挑战，因为其采集值在许多区域数值上趋近于零。这种困难通常随着观测数量、搜索空间维度或约束数量的增加而加剧，导致文献中报告的性能不一致，且多数情况下表现欠佳。在此，我们提出LogEI——一类新的采集函数，其成员与经典对应函数要么具有相同或近似相等的最优值，但数值优化难度显著降低。我们证明了数值病态问题在“经典”解析EI、期望超体积改进（EHVI）及其约束、噪声和并行变体中均有体现，并提出了相应的修正方案以解决这些病态问题。实证结果表明，LogEI类采集函数的成员在优化性能上显著优于其经典对应函数，且令人惊讶地与近期最先进的采集函数性能持平甚至超越，这凸显了文献中数值优化作用被低估的问题。