Bayesian optimization is a sequential procedure for obtaining the global optimum of black-box functions without knowing a priori their true form. Good uncertainty estimates over the shape of the objective function are essential in guiding the optimization process. However, these estimates can be inaccurate if the true objective function violates assumptions made by its model (e.g., Gaussianity). This paper studies which uncertainties are needed in Bayesian optimization models and argues that ideal uncertainties should be calibrated -- i.e., an 80% predictive interval should contain the true outcome 80% of the time. We propose a simple algorithm for enforcing this property and show that it enables Bayesian optimization to arrive at the global optimum in fewer steps. We provide theoretical insights into the role of calibrated uncertainties and demonstrate the improved performance of our method on standard benchmark functions and hyperparameter optimization tasks.

翻译：贝叶斯优化是一种序贯程序，用于在不预先了解黑箱函数真实形式的情况下获取其全局最优解。对目标函数形状的良好不确定性估计对于指导优化过程至关重要。然而，若真实目标函数违反其模型所作假设（例如高斯性），这些估计可能不准确。本文研究贝叶斯优化模型中需要何种不确定性，并论证理想的不确定性应是校准的——即一个80%的预测区间应有80%的概率包含真实结果。我们提出一种强制执行该属性的简单算法，并表明该算法能使贝叶斯优化以更少的步骤到达全局最优解。我们提供了关于校准不确定性作用的理论见解，并在标准基准函数和超参数优化任务上展示了我们方法改进后的性能。