Bayesian optimization (BO) is widely used to optimize black-box functions. It works by first building a surrogate for the objective and quantifying the uncertainty in that surrogate. It then decides where to sample by maximizing an acquisition function defined by the surrogate model. Prior approaches typically use randomly generated raw samples to initialize the acquisition function maximizer. However, this strategy is ill-suited for high-dimensional BO. Given the large regions of high posterior uncertainty in high dimensions, a randomly initialized acquisition function maximizer is likely to focus on areas with high posterior uncertainty, leading to overly exploring areas that offer little gain. This paper provides the first comprehensive empirical study to reveal the importance of the initialization phase of acquisition function maximization. It proposes a better initialization approach by employing multiple heuristic optimizers to leverage the knowledge of already evaluated samples to generate initial points to be explored by an acquisition function maximizer. We evaluate our approach on widely used synthetic test functions and real-world applications. Experimental results show that our techniques, while simple, can significantly enhance the standard BO and outperforms state-of-the-art high-dimensional BO techniques by a large margin in most test cases.

翻译：贝叶斯优化广泛应用于黑箱函数优化。该方法首先为目标函数构建代理模型并量化其不确定性，随后通过最大化由代理模型定义的采集函数确定采样位置。现有方法通常采用随机生成的原始样本初始化采集函数最大化过程，然而该策略并不适用于高维贝叶斯优化。由于高维空间中后验不确定性较大的区域往往范围广阔，随机初始化的采集函数最大化器容易聚焦于高后验不确定性区域，导致过度探索价值有限的区域。本文首次通过系统性实证研究揭示采集函数最大化初始化阶段的关键作用，并提出一种改进的初始化方法：采用多种启发式优化器，利用已评估样本的知识生成待探索的初始点。我们在广泛使用的合成测试函数和实际应用场景中进行了评估，实验结果表明，本文提出的简单技术能够显著提升标准贝叶斯优化性能，并在大多数测试案例中大幅超越当前最先进的高维贝叶斯优化技术。