Bayesian optimization is a principled optimization strategy for a black-box objective function. It shows its effectiveness in a wide variety of real-world applications such as scientific discovery and experimental design. In general, the performance of Bayesian optimization is assessed by regret-based metrics such as instantaneous, simple, and cumulative regrets. These metrics only rely on function evaluations, so that they do not consider geometric relationships between query points and global solutions, or query points themselves. Notably, they cannot discriminate if multiple global solutions are successfully found. Moreover, they do not evaluate Bayesian optimization's abilities to exploit and explore a search space given. To tackle these issues, we propose four new geometric metrics, i.e., precision, recall, average degree, and average distance. These metrics allow us to compare Bayesian optimization algorithms considering the geometry of both query points and global optima, or query points. However, they are accompanied by an extra parameter, which needs to be carefully determined. We therefore devise the parameter-free forms of the respective metrics by integrating out the additional parameter. Finally, we empirically validate that our proposed metrics can provide more convincing interpretation and understanding of Bayesian optimization algorithms from distinct perspectives, compared to the conventional metrics.

翻译：贝叶斯优化是一种针对黑箱目标函数的规范化优化策略，在科学发现和实验设计等众多实际应用中展现出有效性。通常，贝叶斯优化的性能通过基于遗憾的度量评估，例如即时遗憾、简单遗憾和累积遗憾。这些度量仅依赖函数评估值，因此未考虑查询点与全局解之间或查询点本身的几何关系。值得注意的是，它们无法区分是否成功找到多个全局解，也未评估贝叶斯优化对搜索空间的开发与探索能力。为解决这些问题，我们提出四种新的几何度量：精确率、召回率、平均度数和平均距离。这些度量通过考虑查询点与全局最优点（或查询点自身）的几何结构，实现对贝叶斯优化算法的比较。然而，它们伴随一个需谨慎确定的额外参数。为此，我们通过对该额外参数进行积分，推导出各度量的无参数化形式。最后，实验验证表明：与传统度量相比，我们提出的度量能从不同角度为贝叶斯优化算法提供更具说服力的解释与理解。