Bayesian optimization is a methodology for global optimization of unknown and expensive objectives. It combines a surrogate Bayesian regression model with an acquisition function to decide where to evaluate the objective. Typical regression models are given by Gaussian processes with stationary covariance functions. However, these functions are unable to express prior input-dependent information, including possible locations of the optimum. The ubiquity of stationary models has led to the common practice of exploiting prior information via informative mean functions. In this paper, we highlight that these models can perform poorly, especially in high dimensions. We propose novel informative covariance functions for optimization, leveraging nonstationarity to encode preferences for certain regions of the search space and adaptively promote local exploration during optimization. We demonstrate that the proposed functions can increase the sample efficiency of Bayesian optimization in high dimensions, even under weak prior information.

翻译：贝叶斯优化是一种针对未知且昂贵的全局优化方法。它将替代性的贝叶斯回归模型与采集函数相结合，以决定在何处评估目标函数。典型的回归模型由具有平稳协方差函数的高斯过程构成。然而，这些函数无法表达先验的输入相关信息，包括最优点的可能位置。由于平稳模型的广泛使用，通常通过信息性均值函数来利用先验信息。本文强调，这类模型可能表现不佳，尤其是在高维场景中。我们提出了用于优化的新型信息协方差函数，通过非平稳性来编码对搜索空间特定区域的偏好，并在优化过程中自适应地促进局部探索。实验证明，即使在先验信息较弱的情况下，所提出的函数也能提高高维贝叶斯优化的样本效率。