Bayesian optimization is a principled optimization strategy for a black-box objective function. It shows its effectiveness in a wide variety of real-world applications such as scientific discovery and experimental design. In general, the performance of Bayesian optimization is reported through regret-based metrics such as instantaneous, simple, and cumulative regrets. These metrics only rely on function evaluations, so that they do not consider geometric relationships between query points and global solutions, or query points themselves. Notably, they cannot discriminate if multiple global solutions are successfully found. Moreover, they do not evaluate Bayesian optimization's abilities to exploit and explore a search space given. To tackle these issues, we propose four new geometric metrics, i.e., precision, recall, average degree, and average distance. These metrics allow us to compare Bayesian optimization algorithms considering the geometry of both query points and global optima, or query points. However, they are accompanied by an extra parameter, which needs to be carefully determined. We therefore devise the parameter-free forms of the respective metrics by integrating out the additional parameter. Finally, we validate that our proposed metrics can provide more delicate interpretation of Bayesian optimization, on top of assessment via the conventional metrics.

翻译：贝叶斯优化是一种针对黑箱目标函数的原理性优化策略，在科学发现与实验设计等众多现实应用中展现出卓越效能。传统上，贝叶斯优化的性能通常通过基于遗憾的度量进行评估，例如瞬时遗憾、简单遗憾与累积遗憾。这类度量仅依赖于函数评估值，既未考虑查询点与全局解之间的几何关系，也未考虑查询点自身的空间分布特性。尤其值得注意的是，当算法成功找到多个全局解时，此类度量无法进行有效区分。此外，它们亦不能评估贝叶斯优化在给定搜索空间中开发与探索的平衡能力。为应对这些局限，本文提出四种新型几何度量：精确率、召回率、平均度数与平均距离。这些度量使我们能够结合查询点与全局最优解（或查询点之间）的几何特性来比较不同贝叶斯优化算法。然而，这些度量均涉及一个需要审慎确定的额外参数。为此，我们通过对额外参数进行积分处理，构建了各度量的无参数化形式。最终，我们通过实验验证了所提度量能够在传统评估体系之上，为贝叶斯优化的性能提供更精细的解读维度。