Bayesian Optimization (BO), guided by Gaussian process (GP) surrogates, has proven to be an invaluable technique for efficient, high-dimensional, black-box optimization, a critical problem inherent to many applications such as industrial design and scientific computing. Recent contributions have introduced reinforcement learning (RL) to improve the optimization performance on both single function optimization and \textit{few-shot} multi-objective optimization. However, even few-shot techniques fail to exploit similarities shared between closely related objectives. In this paper, we combine recent developments in Deep Kernel Learning (DKL) and attention-based Transformer models to improve the modeling powers of GP surrogates with meta-learning. We propose a novel method for improving meta-learning BO surrogates by incorporating attention mechanisms into DKL, empowering the surrogates to adapt to contextual information gathered during the BO process. We combine this Transformer Deep Kernel with a learned acquisition function trained with continuous Soft Actor-Critic Reinforcement Learning to aid in exploration. This Reinforced Transformer Deep Kernel (RTDK-BO) approach yields state-of-the-art results in continuous high-dimensional optimization problems.

翻译：贝叶斯优化（BO）以高斯过程（GP）代理模型为指导，已被证明是高效、高维黑箱优化的宝贵技术，而这类优化正是工业设计和科学计算等许多应用中固有的关键问题。近期研究引入强化学习（RL）来提升单函数优化和\textit{少样本}多目标优化的性能。然而，即使少样本技术也无法充分利用紧密相关目标之间的相似性。本文结合深度核学习（DKL）和基于注意力的Transformer模型的最新进展，通过元学习增强GP代理的建模能力。我们提出一种改进元学习BO代理模型的新方法，将注意力机制融入DKL，使代理能够适应BO过程中收集的上下文信息。我们将此Transformer深度核与通过连续Soft Actor-Critic强化学习训练的习得采集函数相结合，以辅助探索。这种强化Transformer深度核（RTDK-BO）方法在连续高维优化问题中取得了最先进的结果。