We introduce a framework for benchmarking optimizers according to multiple criteria over various test functions. Based on a recently introduced union-free generic depth function for partial orders/rankings, it fully exploits the ordinal information and allows for incomparability. Our method describes the distribution of all partial orders/rankings, avoiding the notorious shortcomings of aggregation. This permits to identify test functions that produce central or outlying rankings of optimizers and to assess the quality of benchmarking suites.

翻译：我们提出了一种框架，用于根据多个准则在各种测试函数上对优化器进行基准测试。该方法基于近期引入的针对偏序/排序的无并集通用深度函数，充分利用了序数信息并允许不可比性。我们的方法描述了所有偏序/排序的分布，避免了聚合的显著缺陷。这使得能够识别产生优化器中心排序或离群排序的测试函数，并评估基准测试套件的质量。