Bayesian optimization (BO) iteratively fits a Gaussian process (GP) surrogate to accumulated evaluations and selects new queries via an acquisition function such as expected improvement (EI). In practice, BO often concentrates evaluations near the current incumbent, causing the surrogate to become overconfident and to understate predictive uncertainty in the region guiding subsequent decisions. We develop a robust GP-based BO via tempered posterior updates, which downweight the likelihood by a power $α\in (0,1]$ to mitigate overconfidence under local misspecification. We establish cumulative regret bounds for tempered BO under a family of generalized improvement rules, including EI, and show that tempering yields strictly sharper worst-case regret guarantees than the standard posterior $(α=1)$, with the most favorable guarantees occurring near the classical EI choice. Motivated by our theoretic findings, we propose a prequential procedure for selecting $α$ online: it decreases $α$ when realized prediction errors exceed model-implied uncertainty and returns $α$ toward one as calibration improves. Empirical results demonstrate that tempering provides a practical yet theoretically grounded tool for stabilizing BO surrogates under localized sampling.
翻译:贝叶斯优化(BO)通过迭代拟合高斯过程(GP)代理模型以累积评估数据,并借助期望提升(EI)等采集函数选择新的查询点。在实际应用中,BO常将评估点集中在当前最优解附近,导致代理模型过度自信,并在指导后续决策的关键区域低估预测不确定性。本文提出一种基于调温后验更新的鲁棒GP贝叶斯优化方法:通过将似然函数以幂次$α\in (0,1]$进行降权,以缓解局部模型误设下的过度自信问题。我们建立了调温BO在包含EI在内的广义提升规则族下的累积遗憾界,证明调温处理相比标准后验($α=1$)能获得严格更优的最坏情况遗憾保证,且最优保证出现在经典EI选择附近。基于理论发现,我们提出一种在线选择$α$的序贯预测方法:当实际预测误差超过模型隐含的不确定性时降低$α$值,并在模型校准改善时使$α$回归至1。实验结果表明,调温处理为局部采样场景下的BO代理模型稳定提供了一种兼具理论依据与实践价值的工具。