Bayesian optimisation is a sample efficient method for finding a global optimum of expensive black-box objective functions. Historic datasets from related problems can be exploited to help improve performance of Bayesian optimisation by adapting transfer learning methods to various components of the Bayesian optimisation pipeline. In this study we perform an empirical analysis of various ensemble-based transfer learning Bayesian optimisation methods and pipeline components. We expand on previous work in the literature by contributing some specific pipeline components, and three new real-time transfer learning Bayesian optimisation benchmarks. In particular we propose to use a weighting strategy for ensemble surrogate model predictions based on regularised regression with weights constrained to be positive, and a related component for handling the case when transfer learning is not improving Bayesian optimisation performance. We find that in general, two components that help improve transfer learning Bayesian optimisation performance are warm start initialisation and constraining weights used with ensemble surrogate model to be positive.

翻译：贝叶斯优化是一种样本高效的全局优化方法，适用于求解昂贵黑箱目标函数的最优解。通过将迁移学习方法适配至贝叶斯优化流程的各个组件，可利用相关问题的历史数据集提升贝叶斯优化性能。本研究对多种基于集成的迁移学习贝叶斯优化方法及流程组件进行了实证分析。我们在现有文献基础上扩展了研究内容，贡献了若干特定流程组件及三个新的实时迁移学习贝叶斯优化基准测试。特别地，我们提出采用基于正则化回归的加权策略处理集成代理模型预测问题（权重约束为正），并设计了相关组件以应对迁移学习未提升贝叶斯优化性能的情况。研究发现，有助于提升迁移学习贝叶斯优化性能的两个关键组件是：热启动初始化策略以及对集成代理模型使用正权重约束。