Many problems in science and engineering involve optimizing an expensive black-box function over a high-dimensional space. For such black-box optimization (BBO) problems, we typically assume a small budget for online function evaluations, but also often have access to a fixed, offline dataset for pretraining. Prior approaches seek to utilize the offline data to approximate the function or its inverse but are not sufficiently accurate far from the data distribution. We propose BONET, a generative framework for pretraining a novel black-box optimizer using offline datasets. In BONET, we train an autoregressive model on fixed-length trajectories derived from an offline dataset. We design a sampling strategy to synthesize trajectories from offline data using a simple heuristic of rolling out monotonic transitions from low-fidelity to high-fidelity samples. Empirically, we instantiate BONET using a causally masked Transformer and evaluate it on Design-Bench, where we rank the best on average, outperforming state-of-the-art baselines.

翻译：科学与工程中的许多问题涉及在高维空间中优化代价高昂的黑箱函数。针对此类黑箱优化问题，我们通常假定在线函数评估的预算有限，但往往也能获得固定离线数据集用于预训练。现有方法试图利用离线数据近似函数或其逆映射，但在远离数据分布的区域精度不足。我们提出BONET——一种基于离线数据集对新型黑箱优化器进行预训练的生成式框架。在BONET中，我们训练一个自回归模型处理源自离线数据集的定长轨迹。通过设计采样策略，利用从低保真样本到高保真样本单调递进展开的启发式方法，从离线数据合成轨迹。实验表明，我们采用因果掩码Transformer实现BONET，并在Design-Bench基准上取得平均最优排名，性能超越当前最先进的基线方法。