Accurate uncertainty estimates are important in sequential model-based decision-making tasks such as Bayesian optimization. However, these estimates can be imperfect if the data violates assumptions made by the model (e.g., Gaussianity). This paper studies which uncertainties are needed in model-based decision-making and in Bayesian optimization, and argues that uncertainties can benefit from calibration -- i.e., an 80% predictive interval should contain the true outcome 80% of the time. Maintaining calibration, however, can be challenging when the data is non-stationary and depends on our actions. We propose using simple algorithms based on online learning to provably maintain calibration on non-i.i.d. data, and we show how to integrate these algorithms in Bayesian optimization with minimal overhead. Empirically, we find that calibrated Bayesian optimization converges to better optima in fewer steps, and we demonstrate improved performance on standard benchmark functions and hyperparameter optimization tasks.

翻译：在贝叶斯优化等基于序列模型的决策任务中，准确的概率不确定性估计至关重要。然而，当数据违背模型假设（例如高斯性）时，这些估计可能失效。本文研究了基于模型的决策与贝叶斯优化中所需的不确定性类型，论证了校准对不确定性估计的增益作用——即80%的预测区间应实际包含80%的真实观测结果。然而，当数据呈现非平稳特性且受决策行为影响时，维持校准状态面临挑战。我们提出基于在线学习的轻量级算法，可在非独立同分布数据上可证明地维持校准特性，并展示了如何以最小计算开销将这些算法集成至贝叶斯优化框架。实验表明，经过校准的贝叶斯优化能以更少步骤收敛至更优解，在标准基准函数测试与超参数优化任务中均表现出性能提升。