Bayesian Optimization (BO) is a powerful method for optimizing black-box functions by combining prior knowledge with ongoing function evaluations. BO constructs a probabilistic surrogate model of the objective function given the covariates, which is in turn used to inform the selection of future evaluation points through an acquisition function. For smooth continuous search spaces, Gaussian Processes (GPs) are commonly used as the surrogate model as they offer analytical access to posterior predictive distributions, thus facilitating the computation and optimization of acquisition functions. However, in complex scenarios involving optimizations over categorical or mixed covariate spaces, GPs may not be ideal. This paper introduces Simulation Based Bayesian Optimization (SBBO) as a novel approach to optimizing acquisition functions that only requires \emph{sampling-based} access to posterior predictive distributions. SBBO allows the use of surrogate probabilistic models tailored for combinatorial spaces with discrete variables. Any Bayesian model in which posterior inference is carried out through Markov chain Monte Carlo can be selected as the surrogate model in SBBO. In applications involving combinatorial optimization, we demonstrate empirically the effectiveness of SBBO method using various choices of surrogate models.

翻译：贝叶斯优化（BO）是一种通过结合先验知识与持续的函数评估来优化黑箱函数的强大方法。BO根据协变量构建目标函数的概率代理模型，进而通过采集函数指导未来评估点的选择。对于光滑连续搜索空间，高斯过程（GPs）常用作代理模型，因其能提供后验预测分布的解析形式，从而便于采集函数的计算与优化。然而，在涉及分类变量或混合协变量空间的复杂优化场景中，高斯过程可能并非理想选择。本文提出基于仿真的贝叶斯优化（SBBO）作为一种优化采集函数的新方法，该方法仅需对后验预测分布进行基于采样的访问。SBBO允许使用专为包含离散变量的组合空间定制的概率代理模型。任何通过马尔可夫链蒙特卡洛进行后验推断的贝叶斯模型均可选作SBBO的代理模型。在涉及组合优化的应用中，我们通过实验证明采用不同代理模型选择的SBBO方法的有效性。