Bayesian optimization devolves the global optimization of a costly objective function to the global optimization of a sequence of acquisition functions. This inner-loop optimization can be catastrophically difficult if it involves posterior sample paths, especially in higher dimensions. We introduce an efficient global optimization strategy for posterior samples based on global rootfinding. It provides gradient-based optimizers with two sets of judiciously selected starting points, designed to combine exploration and exploitation. The number of starting points can be kept small without sacrificing optimization quality. Remarkably, even with just one point from each set, the global optimum is discovered most of the time. The algorithm scales practically linearly to high dimensions, breaking the curse of dimensionality. For Gaussian process Thompson sampling (GP-TS), we demonstrate remarkable improvement in both inner- and outer-loop optimization, surprisingly outperforming alternatives like EI and GP-UCB in most cases. Our approach also improves the performance of other posterior sample-based acquisition functions, such as variants of entropy search. Furthermore, we propose a sample-average formulation of GP-TS, which has a parameter to explicitly control exploitation and can be computed at the cost of one posterior sample. Our implementation is available at https://github.com/UQUH/TSRoots .

翻译：贝叶斯优化将代价高昂的目标函数的全局优化问题转化为一系列采集函数的全局优化问题。若该内层优化涉及后验样本路径，则优化过程可能变得极其困难，尤其是在高维情形下。本文提出一种基于全局根求解的高效后验样本全局优化策略。该策略为基于梯度的优化器提供两组经过精心选择的起始点，旨在平衡探索与利用。即使起始点数量较少，仍能保持优化质量。值得注意的是，即使每组仅取一个起始点，在大多数情况下仍能发现全局最优解。该算法在实际应用中可近似线性地扩展至高维空间，从而突破维度灾难的限制。针对高斯过程汤普森采样（GP-TS），我们展示了该方法在内层与外层优化中的显著改进，在大多数情况下甚至意外地超越了如EI和GP-UCB等替代方法。本方法同样提升了其他基于后验样本的采集函数（如各类熵搜索变体）的性能。此外，我们提出一种GP-TS的样本平均形式，该形式通过显式参数控制利用程度，且计算成本仅相当于一次后验采样。算法实现详见 https://github.com/UQUH/TSRoots。