Bayesian optimization (BO) is a popular approach for optimizing expensive-to-evaluate black-box objective functions. An important challenge in BO is its application to high-dimensional search spaces due in large part to the curse of dimensionality. One way to overcome this challenge is to focus on local BO methods that aim to efficiently learn gradients, which have shown strong empirical performance on a variety of high-dimensional problems including policy search in reinforcement learning (RL). However, current local BO methods assume access to only a single high-fidelity information source whereas, in many engineering and control problems, one has access to multiple cheaper approximations of the objective. We propose a novel algorithm, Cost-Aware Gradient Entropy Search (CAGES), for local BO of multi-fidelity black-box functions. CAGES makes no assumption about the relationship between different information sources, making it more flexible than other multi-fidelity methods. It also employs a new type of information-theoretic acquisition function, which enables systematic identification of samples that maximize the information gain about the unknown gradient per cost of the evaluation. We demonstrate CAGES can achieve significant performance improvements compared to other state-of-the-art methods on a variety of synthetic and benchmark RL problems.

翻译：贝叶斯优化（BO）是一种针对昂贵黑箱目标函数优化的主流方法。BO面临的重要挑战之一是其在高维搜索空间中的应用，这主要源于维度灾难。克服该挑战的一种方法是聚焦于旨在高效学习梯度的局部BO方法，这类方法已在包括强化学习（RL）策略搜索在内的多种高维问题上展现出强劲的实证性能。然而，现有局部BO方法仅能获取单一高保真信息源，而在许多工程与控制问题中，研究者可获取多个更廉价的目标函数近似源。我们提出了一种新颖算法——代价感知梯度熵搜索（CAGES），用于多保真黑箱函数的局部BO。CAGES无需假设不同信息源之间的关系，使其比其他多保真方法更具灵活性。该方法还采用了一种新型信息论采集函数，能够系统地识别每次评估代价下使未知梯度信息增益最大化的样本。我们证明，在多种合成与基准RL问题上，CAGES相较于其他先进方法能实现显著的性能提升。