Bayesian optimisation (BO) uses probabilistic surrogate models - usually Gaussian processes (GPs) - for the optimisation of expensive black-box functions. At each BO iteration, the GP hyperparameters are fit to previously-evaluated data by maximising the marginal likelihood. However, this fails to account for uncertainty in the hyperparameters themselves, leading to overconfident model predictions. This uncertainty can be accounted for by taking the Bayesian approach of marginalising out the model hyperparameters. We investigate whether a fully-Bayesian treatment of the Gaussian process hyperparameters in BO (FBBO) leads to improved optimisation performance. Since an analytic approach is intractable, we compare FBBO using three approximate inference schemes to the maximum likelihood approach, using the Expected Improvement (EI) and Upper Confidence Bound (UCB) acquisition functions paired with ARD and isotropic Matern kernels, across 15 well-known benchmark problems for 4 observational noise settings. FBBO using EI with an ARD kernel leads to the best performance in the noise-free setting, with much less difference between combinations of BO components when the noise is increased. FBBO leads to over-exploration with UCB, but is not detrimental with EI. Therefore, we recommend that FBBO using EI with an ARD kernel as the default choice for BO.

翻译：贝叶斯优化 (BO) 使用概率替代模型(通常是高斯进程(GPs) ) 来优化昂贵黑盒功能。 在BO 的每一次循环中, GP 超参数都适合先前评估的数据, 使边际可能性最大化。 但是, 这不能说明超参数本身的不确定性, 导致过度信任模型预测。 这种不确定性可以通过采取巴伊西亚方法将模型超参数边缘化来解释。 我们调查在BO( FBBO) 中对高山进程超参数进行完全的巴耶斯处理是否导致优化性能的改善。 由于分析方法非常复杂, 我们用三种近似推导法将FBBO与最大可能性方法进行比较, 使用预期改进(EI) 和高信任模型(UCB) 获得功能, 与ARD和偏向偏差(I) 15个众所周知的基准问题。 使用EBO 与 EI 和 EBEO 相比, 与 EBI 和 EB BO 混合, 与E-B 与 EB 不同,, 与E-B 与E- BO 的 将 与E-B 与E- BI 的 的 的更低的混合比。