Bayesian optimization over the latent spaces of deep autoencoder models (DAEs) has recently emerged as a promising new approach for optimizing challenging black-box functions over structured, discrete, hard-to-enumerate search spaces (e.g., molecules). Here the DAE dramatically simplifies the search space by mapping inputs into a continuous latent space where familiar Bayesian optimization tools can be more readily applied. Despite this simplification, the latent space typically remains high-dimensional. Thus, even with a well-suited latent space, these approaches do not necessarily provide a complete solution, but may rather shift the structured optimization problem to a high-dimensional one. In this paper, we propose LOL-BO, which adapts the notion of trust regions explored in recent work on high-dimensional Bayesian optimization to the structured setting. By reformulating the encoder to function as both an encoder for the DAE globally and as a deep kernel for the surrogate model within a trust region, we better align the notion of local optimization in the latent space with local optimization in the input space. LOL-BO achieves as much as 20 times improvement over state-of-the-art latent space Bayesian optimization methods across six real-world benchmarks, demonstrating that improvement in optimization strategies is as important as developing better DAE models.

翻译：深度自编码器（DAE）模型潜变量空间上的贝叶斯优化，最近成为一种前景广阔的新方法，用于在结构化、离散且难以枚举的搜索空间（如分子）中优化具有挑战性的黑箱函数。该方法通过将输入映射到连续潜变量空间，显著简化了搜索空间，使得常用的贝叶斯优化工具能更便捷地应用。然而，尽管实现了这种简化，潜变量空间通常仍是高维的。因此，即便拥有合适的潜变量空间，这些方法未必能提供完整的解决方案，反而可能将结构化优化问题转化为高维优化问题。本文提出了LOL-BO，它将近期高维贝叶斯优化研究中探索的置信域概念适配到结构化场景中。通过重新设计编码器，使其既作为DAE的全局编码器，又作为置信区间内代理模型的深度核函数，我们更好地对齐了潜变量空间中的局部优化与输入空间中的局部优化。在六个真实世界基准测试中，LOL-BO相比最先进的潜变量空间贝叶斯优化方法实现了高达20倍的性能提升，这表明优化策略的改进与开发更优的DAE模型同等重要。