A wide spectrum of design and decision problems, including parameter tuning, A/B testing and drug design, intrinsically are instances of black-box optimization. Bayesian optimization (BO) is a powerful tool that models and optimizes such expensive "black-box" functions. However, at the beginning of optimization, vanilla Bayesian optimization methods often suffer from slow convergence issue due to inaccurate modeling based on few trials. To address this issue, researchers in the BO community propose to incorporate the spirit of transfer learning to accelerate optimization process, which could borrow strength from the past tasks (source tasks) to accelerate the current optimization problem (target task). This survey paper first summarizes transfer learning methods for Bayesian optimization from four perspectives: initial points design, search space design, surrogate model, and acquisition function. Then it highlights its methodological aspects and technical details for each approach. Finally, it showcases a wide range of applications and proposes promising future directions.

翻译：广泛的设计与决策问题，包括参数调优、A/B测试和药物设计，本质上都是黑箱优化的实例。贝叶斯优化（BO）是一种强大的工具，用于建模和优化这类昂贵的"黑箱"函数。然而，在优化初期，标准的贝叶斯优化方法常因基于少量试验的不准确建模而面临收敛缓慢的问题。为解决这一问题，贝叶斯优化领域的研究人员提出融入迁移学习的思想来加速优化过程，即通过借鉴历史任务（源任务）的优势来加速当前优化问题（目标任务）。本综述首先从四个角度总结了贝叶斯优化中的迁移学习方法：初始点设计、搜索空间设计、代理模型和采集函数；随后重点阐述了每种方法的方法论特点和技术细节；最后展示了广泛的应用场景，并提出了有前景的未来研究方向。