Bayesian optimization relies on iteratively constructing and optimizing an acquisition function. The latter turns out to be a challenging, non-convex optimization problem itself. Despite the relative importance of this step, most algorithms employ sampling- or gradient-based methods, which do not provably converge to global optima. This work investigates mixed-integer programming (MIP) as a paradigm for global acquisition function optimization. Specifically, our Piecewise-linear Kernel Mixed Integer Quadratic Programming (PK-MIQP) formulation introduces a piecewise-linear approximation for Gaussian process kernels and admits a corresponding MIQP representation for acquisition functions. The proposed method is applicable to uncertainty-based acquisition functions for any stationary or dot-product kernel. We analyze the theoretical regret bounds of the proposed approximation, and empirically demonstrate the framework on synthetic functions, constrained benchmarks, and a hyperparameter tuning task.

翻译：贝叶斯优化依赖于迭代构建和优化获取函数。后者本身是一个具有挑战性的非凸优化问题。尽管这一步骤相对重要，大多数算法仍采用基于采样或梯度的方法，这些方法无法保证收敛到全局最优解。本研究探讨了将混合整数规划（MIP）作为全局获取函数优化的范式。具体而言，我们提出的分段线性核混合整数二次规划（PK-MIQP）公式引入了高斯过程核函数的分段线性近似，并允许获取函数采用相应的MIQP表示。该方法适用于任何平稳核或点积核的基于不确定性的获取函数。我们分析了所提出近似的理论遗憾界，并在合成函数、约束基准测试和超参数调优任务上实证验证了该框架。