Gaussian processes are the model of choice in Bayesian optimization and active learning. Yet, they are highly dependent on cleverly chosen hyperparameters to reach their full potential, and little effort is devoted to finding good hyperparameters in the literature. We demonstrate the impact of selecting good hyperparameters for GPs and present two acquisition functions that explicitly prioritize hyperparameter learning. Statistical distance-based Active Learning (SAL) considers the average disagreement between samples from the posterior, as measured by a statistical distance. SAL outperforms the state-of-the-art in Bayesian active learning on several test functions. We then introduce Self-Correcting Bayesian Optimization (SCoreBO), which extends SAL to perform Bayesian optimization and active learning simultaneously. SCoreBO learns the model hyperparameters at improved rates compared to vanilla BO, while outperforming the latest Bayesian optimization methods on traditional benchmarks. Moreover, we demonstrate the importance of self-correction on atypical Bayesian optimization tasks.

翻译：高斯过程是贝叶斯优化和主动学习中的首选模型。然而，它们高度依赖于巧妙选择的超参数以发挥全部潜力，而文献中却鲜有致力于寻找良好超参数的工作。我们展示了为高斯过程选择良好超参数的影响，并提出了两种显式优先考虑超参数学习的采集函数。基于统计距离的主动学习（SAL）考虑后验样本间的平均差异，该差异由统计距离度量。SAL在多个测试函数上的贝叶斯主动学习性能优于现有最优方法。随后，我们引入了自校正贝叶斯优化（SCoreBO），它将SAL扩展以同时执行贝叶斯优化和主动学习。与标准贝叶斯优化相比，SCoreBO能以更快的速度学习模型超参数，同时在传统基准测试中优于最新的贝叶斯优化方法。此外，我们还证明了自校正在非典型贝叶斯优化任务中的重要性。