For a wide range of applications the structure of systems like Neural Networks or complex simulations, is unknown and approximation is costly or even impossible. Black-box optimization seeks to find optimal (hyper-) parameters for these systems such that a pre-defined objective function is minimized. Polynomial-Model-Based Optimization (PMBO) is a novel blackbox optimizer that finds the minimum by fitting a polynomial surrogate to the objective function. Motivated by Bayesian optimization the model is iteratively updated according to the acquisition function Expected Improvement, thus balancing the exploitation and exploration rate and providing an uncertainty estimate of the model. PMBO is benchmarked against other state-of-the-art algorithms for a given set of artificial, analytical functions. PMBO competes successfully with those algorithms and even outperforms all of them in some cases. As the results suggest, we believe PMBO is the pivotal choice for solving blackbox optimization tasks occurring in a wide range of disciplines.
翻译:对于广泛的应用场景,如神经网络或复杂仿真等系统,其结构未知且近似成本高昂甚至不可行。黑箱优化旨在为这些系统寻找最优(超)参数,以使预定义的目标函数最小化。基于多项式模型的黑箱优化(PMBO)是一种新颖的黑箱优化器,它通过拟合多项式代理模型来求解目标函数最小值。受贝叶斯优化启发,该模型根据采集函数“期望改进”迭代更新,从而平衡探索与利用速率,并提供模型的不确定性估计。在给定的人工分析函数集合上,PMBO 与其他主流算法进行了基准测试。结果表明,PMBO 能成功与这些算法竞争,甚至在某些情况下超越所有算法。基于这些结果,我们认为 PMBO 是解决跨学科领域广泛出现的黑箱优化任务的关键选择。