Unconstrained global optimisation aims to optimise expensive-to-evaluate black-box functions without gradient information. Bayesian optimisation, one of the most well-known techniques, typically employs Gaussian processes as surrogate models, leveraging their probabilistic nature to balance exploration and exploitation. However, Gaussian processes become computationally prohibitive in high-dimensional spaces. Recent alternatives, based on inverse distance weighting (IDW) and radial basis functions (RBFs), offer competitive, computationally lighter solutions. Despite their efficiency, both traditional global and Bayesian optimisation strategies suffer from the myopic nature of their acquisition functions, which focus solely on immediate improvement neglecting future implications of the sequential decision making process. Nonmyopic acquisition functions devised for the Bayesian setting have shown promise in improving long-term performance. Yet, their use in deterministic strategies with IDW and RBF remains unexplored. In this work, we introduce novel nonmyopic acquisition strategies tailored to IDW- and RBF-based global optimisation. Specifically, we develop dynamic programming-based paradigms, including rollout and multi-step scenario-based optimisation schemes, to enable lookahead acquisition. These methods optimise a sequence of query points over a horizon (instead of only at the next step) by predicting the evolution of the surrogate model, inherently managing the exploration-exploitation trade-off in a systematic way via optimisation techniques. The proposed approach represents a significant advance in extending nonmyopic acquisition principles, previously confined to Bayesian optimisation, to the deterministic framework. Empirical results on synthetic and hyperparameter tuning benchmark problems demonstrate that these nonmyopic methods outperform conventional myopic approaches.
翻译:无约束全局优化的目标是在没有梯度信息的情况下优化评估成本高昂的黑箱函数。贝叶斯优化作为最著名的技术之一,通常采用高斯过程作为代理模型,利用其概率特性来平衡探索与利用。然而,高斯过程在高维空间中计算成本过高。近年来基于反距离加权(IDW)和径向基函数(RBF)的替代方法提供了计算更轻量且具有竞争力的解决方案。尽管这些方法效率较高,但传统的全局优化和贝叶斯优化策略均受限于其采集函数的近视特性:这些函数仅关注即时改进,而忽略了序贯决策过程的未来影响。为贝叶斯框架设计的非近视采集函数在提升长期性能方面已显示出潜力。然而,在基于IDW和RBF的确定性策略中,其应用尚未得到探索。本研究针对基于IDW和RBF的全局优化提出了新颖的非近视采集策略。具体而言,我们开发了基于动态规划的范式,包括滚动优化和多步场景优化方案,以实现前瞻性采集。这些方法通过预测代理模型的演化,在优化时域内(而非仅下一步)优化一系列查询点,从而通过优化技术以系统化的方式内在管理探索与利用的权衡。所提出的方法代表了将先前局限于贝叶斯优化的非近视采集原则扩展至确定性框架的重要进展。在合成问题和超参数调优基准问题上的实证结果表明,这些非近视方法优于传统的近视方法。