Gaussian Process Upper Confidence Bound (GP-UCB) is one of the most popular methods for optimizing black-box functions with noisy observations, due to its simple structure and superior performance. Its empirical successes lead to a natural, yet unresolved question: Is GP-UCB regret optimal? In this paper, we offer the first generally affirmative answer to this important open question in the Bayesian optimization literature. We establish new upper bounds on both the simple and cumulative regret of GP-UCB when the objective function to optimize admits certain smoothness property. These upper bounds match the known minimax lower bounds (up to logarithmic factors independent of the feasible region's dimensionality) for optimizing functions with the same smoothness. Intriguingly, our findings indicate that, with the same level of exploration, GP-UCB can simultaneously achieve optimality in both simple and cumulative regret. The crux of our analysis hinges on a refined uniform error bound for online estimation of functions in reproducing kernel Hilbert spaces. This error bound, which we derive from empirical process theory, is of independent interest, and its potential applications may reach beyond the scope of this study.

翻译：高斯过程上置信界（GP-UCB）因其结构简单和性能优越，成为带噪声黑箱函数优化中最流行的方法之一。其实验成功引出了一个自然但尚未解决的问题：GP-UCB是否具有遗憾最优性？本文首次对贝叶斯优化文献中这一重要开放问题给出了普遍性肯定答案。当待优化目标函数具有特定光滑性时，我们建立了GP-UCB在简单遗憾和累积遗憾上的新上界。这些上界与已知的（与可行域维度无关的对数因子内）优化相同光滑性函数的极小极大下界相匹配。有趣的是，我们的发现表明，在相同探索水平下，GP-UCB能够同时实现简单遗憾和累积遗憾的最优性。分析的核心依赖于再生核希尔伯特空间中函数在线估计的精细化一致误差界。该误差界源自经验过程理论，具有独立研究价值，其潜在应用可能超越本研究范畴。