The Expected Improvement (EI) method, proposed by Jones et al. (1998), is a widely-used Bayesian optimization method, which makes use of a fitted Gaussian process model for efficient black-box optimization. However, one key drawback of EI is that it is overly greedy in exploiting the fitted Gaussian process model for optimization, which results in suboptimal solutions even with large sample sizes. To address this, we propose a new hierarchical EI (HEI) framework, which makes use of a hierarchical Gaussian process model. HEI preserves a closed-form acquisition function, and corrects the over-greediness of EI by encouraging exploration of the optimization space. We then introduce hyperparameter estimation methods which allow HEI to mimic a fully Bayesian optimization procedure, while avoiding expensive Markov-chain Monte Carlo sampling steps. We prove the global convergence of HEI over a broad function space, and establish near-minimax convergence rates under certain prior specifications. Numerical experiments show the improvement of HEI over existing Bayesian optimization methods, for synthetic functions and a semiconductor manufacturing optimization problem.

翻译：期望改进方法（EI，Jones等人，1998）是一种广泛使用的贝叶斯优化方法，它利用拟合的高斯过程模型实现高效的黑箱优化。然而，EI的一个关键缺点是它在优化过程中过度贪婪地利用拟合的高斯过程模型，即使在大样本量下也会导致次优解。为了解决这一问题，我们提出了一种新的分层EI（HEI）框架，该框架利用分层高斯过程模型。HEI保留了闭合形式的采集函数，并通过鼓励探索优化空间来修正EI的过度贪婪性。随后，我们引入了超参数估计方法，使HEI能够模拟完全的贝叶斯优化过程，同时避免昂贵的马尔可夫链蒙特卡洛采样步骤。我们在广泛的函数空间上证明了HEI的全局收敛性，并在特定先验设定下建立了近极小化最优收敛速率。数值实验表明，在合成函数和半导体制造优化问题上，HEI优于现有的贝叶斯优化方法。