Efficient global optimization (EGO) is one of the most widely used noise-free Bayesian optimization algorithms.It comprises the Gaussian process (GP) surrogate model and expected improvement (EI) acquisition function. In practice, when EGO is applied, a scalar matrix of a small positive value (also called a nugget or jitter) is usually added to the covariance matrix of the deterministic GP to improve numerical stability. We refer to this EGO with a positive nugget as the practical EGO. Despite its wide adoption and empirical success, to date, cumulative regret bounds for practical EGO have yet to be established. In this paper, we present for the first time the cumulative regret upper bound of practical EGO. In particular, we show that practical EGO has sublinear cumulative regret bounds and thus is a no-regret algorithm for commonly used kernels including the squared exponential (SE) and Matérn kernels ($ν>\frac{1}{2}$). Moreover, we analyze the effect of the nugget on the regret bound and discuss the theoretical implication on its choice. Numerical experiments are conducted to support and validate our findings.

翻译：高效全局优化（EGO）是最广泛使用的无噪声贝叶斯优化算法之一。该算法包含高斯过程（GP）替代模型和期望改进（EI）采集函数。实际应用EGO时，通常会在确定性GP的协方差矩阵中添加一个由小正数构成的标量矩阵（也称为nugget或jitter），以提升数值稳定性。我们将这种带有正nugget的EGO称为实用EGO。尽管实用EGO被广泛采用且取得了显著经验成果，但至今尚未建立其累积遗憾界限。本文首次提出了实用EGO的累积遗憾上界。具体而言，我们证明实用EGO具有次线性累积遗憾界，因此对于平方指数（SE）核和Matérn核（ν>1/2）等常用核函数而言，这是一种零遗憾算法。此外，我们分析了nugget对遗憾界的影响，并讨论了其在理论层面上对nugget选择的启示。通过数值实验对研究结论进行了支撑和验证。