Expected Improvement (EI) is arguably the most popular acquisition function in Bayesian optimization and has found countless successful applications, but its performance is often exceeded by that of more recent methods. Notably, EI and its variants, including for the parallel and multi-objective settings, are challenging to optimize because their acquisition values vanish numerically in many regions. This difficulty generally increases as the number of observations, dimensionality of the search space, or the number of constraints grow, resulting in performance that is inconsistent across the literature and most often sub-optimal. Herein, we propose LogEI, a new family of acquisition functions whose members either have identical or approximately equal optima as their canonical counterparts, but are substantially easier to optimize numerically. We demonstrate that numerical pathologies manifest themselves in "classic" analytic EI, Expected Hypervolume Improvement (EHVI), as well as their constrained, noisy, and parallel variants, and propose corresponding reformulations that remedy these pathologies. Our empirical results show that members of the LogEI family of acquisition functions substantially improve on the optimization performance of their canonical counterparts and surprisingly, are on par with or exceed the performance of recent state-of-the-art acquisition functions, highlighting the understated role of numerical optimization in the literature.

翻译：期望改进（EI）可以说是贝叶斯优化中最流行的采集函数，并已成功应用于无数场景，但其性能往往被近年来的新方法所超越。值得注意的是，EI及其变体（包括并行和多目标场景下的变体）的优化具有挑战性，因为其采集值在许多区域会数值趋零。这种困难通常随着观测数据量、搜索空间维度或约束条件的增加而加剧，导致文献中报道的性能表现不一致且大多次优。本文提出LogEI——一个全新的采集函数族，其成员要么与经典对应函数具有完全相同的极值点，要么具有近似相等的极值点，但在数值优化上显著简化。我们证明了数值病态问题同时存在于"经典"解析EI、期望超体积改进（EHVI）及其约束、噪声和并行变体中，并提出了相应的重构公式以消除这些病态特性。实验结果表明，LogEI族采集函数成员在优化性能上显著优于其经典对应函数，并且令人惊讶的是，其性能与当前最先进的采集函数相当甚至更优，这凸显了文献中长期被低估的数值优化作用。