Bayesian optimization (BO) is a popular black-box function optimization method, which makes sequential decisions based on a Bayesian model, typically a Gaussian process (GP), of the function. To ensure the quality of the model, transfer learning approaches have been developed to automatically design GP priors by learning from observations on "training" functions. These training functions are typically required to have the same domain as the "test" function (black-box function to be optimized). In this paper, we introduce MPHD, a model pre-training method on heterogeneous domains, which uses a neural net mapping from domain-specific contexts to specifications of hierarchical GPs. MPHD can be seamlessly integrated with BO to transfer knowledge across heterogeneous search spaces. Our theoretical and empirical results demonstrate the validity of MPHD and its superior performance on challenging black-box function optimization tasks.

翻译：贝叶斯优化（BO）是一种流行的黑箱函数优化方法，其基于贝叶斯模型（通常为高斯过程，GP）对函数进行序贯决策。为确保模型质量，研究者开发了迁移学习方法，通过从"训练"函数的观测中自动学习高斯过程的先验设计。这些训练函数通常需与"测试"函数（待优化的黑箱函数）具有相同定义域。本文提出MPHD方法——一种面向异构定义域的模型预训练方法，该方法利用神经网络将领域特定上下文映射为层次化高斯过程的规范参数。MPHD可无缝集成至贝叶斯优化中，实现跨异构搜索空间的知识迁移。理论分析与实验结果表明，MPHD具有有效性，并在挑战性黑箱函数优化任务中展现出优越性能。