Model selection is an integral problem of model based optimization techniques such as Bayesian optimization (BO). Current approaches often treat model selection as an estimation problem, to be periodically updated with observations coming from the optimization iterations. In this paper, we propose an alternative way to achieve both efficiently. Specifically, we propose a novel way of integrating model selection and BO for the single goal of reaching the function optima faster. The algorithm moves back and forth between BO in the model space and BO in the function space, where the goodness of the recommended model is captured by a score function and fed back, capturing how well the model helped convergence in the function space. The score function is derived in such a way that it neutralizes the effect of the moving nature of the BO in the function space, thus keeping the model selection problem stationary. This back and forth leads to quick convergence for both model selection and BO in the function space. In addition to improved sample efficiency, the framework outputs information about the black-box function. Convergence is proved, and experimental results show significant improvement compared to standard BO.

翻译：模型选择是基于模型优化技术（如贝叶斯优化）的核心问题。现有方法通常将模型选择视为估计问题，通过优化迭代中的观测数据定期更新。本文提出了一种实现两者高效协同的新思路。具体而言，我们提出了一种将模型选择与贝叶斯优化相融合的新方法，以更快达到函数最优值为唯一目标。该算法在模型空间的贝叶斯优化与函数空间的贝叶斯优化之间交替进行，其中推荐模型的质量通过评分函数捕获并反馈，量化模型在函数空间中促进收敛的有效性。评分函数的推导方式使其能抵消函数空间贝叶斯优化动态变化的影响，从而保持模型选择问题的平稳性。这种交替机制使模型选择与函数空间中的贝叶斯优化均能快速收敛。除提升样本效率外，该框架还能输出黑箱函数的相关信息。我们证明了其收敛性，实验结果表明相较于标准贝叶斯优化，该方法具有显著改进。