Accurate uncertainty estimates are important in sequential model-based decision-making tasks such as Bayesian optimization. However, these estimates can be imperfect if the data violates assumptions made by the model (e.g., Gaussianity). This paper studies which uncertainties are needed in model-based decision-making and in Bayesian optimization, and argues that uncertainties can benefit from calibration -- i.e., an 80% predictive interval should contain the true outcome 80% of the time. Maintaining calibration, however, can be challenging when the data is non-stationary and depends on our actions. We propose using simple algorithms based on online learning to provably maintain calibration on non-i.i.d. data, and we show how to integrate these algorithms in Bayesian optimization with minimal overhead. Empirically, we find that calibrated Bayesian optimization converges to better optima in fewer steps, and we demonstrate improved performance on standard benchmark functions and hyperparameter optimization tasks.

翻译：精确的不确定性估计在序列化模型驱动决策任务（如贝叶斯优化）中至关重要。然而，当数据违反模型假设（如高斯性假设）时，这些估计可能不完善。本文研究模型驱动决策与贝叶斯优化中所需的不确定性类型，并论证不确定性可从校准时受益——即80%的预测区间应包含真实结果的80%概率。然而，当数据呈现非平稳性且与决策行为相关时，维持校准性面临挑战。我们提出基于在线学习的简单算法，可证明其在非独立同分布数据上维持校准性，并展示如何以最小开销将这些算法集成到贝叶斯优化中。实验表明，校准后的贝叶斯优化能以更少迭代步数收敛至更优解，并在标准基准函数与超参数优化任务中展现出更优性能。