The goal of Multi-task Bayesian Optimization (MBO) is to minimize the number of queries required to accurately optimize a target black-box function, given access to offline evaluations of other auxiliary functions. When offline datasets are large, the scalability of prior approaches comes at the expense of expressivity and inference quality. We propose JUMBO, an MBO algorithm that sidesteps these limitations by querying additional data based on a combination of acquisition signals derived from training two Gaussian Processes (GP): a cold-GP operating directly in the input domain and a warm-GP that operates in the feature space of a deep neural network pretrained using the offline data. Such a decomposition can dynamically control the reliability of information derived from the online and offline data and the use of pretrained neural networks permits scalability to large offline datasets. Theoretically, we derive regret bounds for JUMBO and show that it achieves no-regret under conditions analogous to GP-UCB (Srinivas et. al. 2010). Empirically, we demonstrate significant performance improvements over existing approaches on two real-world optimization problems: hyper-parameter optimization and automated circuit design.

翻译：多功能巴伊萨最佳化(MBO)的目标是尽量减少准确优化目标黑盒功能所需的查询数量,以获得对其它辅助功能的离线评价。 当离线数据集庞大时,先前方法的可缩放性会损害表达性和推断质量。 我们提议JUMBO, 一种MBO算法,根据培训两个Gaussian process(GP)产生的获取信号组合来绕过这些限制,查询额外数据:一个在输入域直接运行的冷气GP和一个在使用离线数据的深神经网络特征空间运行的暖气GP。这种分解能够动态地控制从在线和离线数据获取的信息的可靠性,而使用预先培训的神经网络则允许对大型离线数据集进行缩放。从理论上讲,我们对JUMBO在类似于GP-UCB(Snivas et. al. 2010)的条件下实现无报复性约束,并显示它在使用深神经网络的特征空间中预先训练。这种分流式分析能够动态地控制从在线和离线数据中获得的信息。 我们展示了在现实最佳优化的两种方式。