We introduce a framework for benchmarking optimizers according to multiple criteria over various test functions. Based on a recently introduced union-free generic depth function for partial orders/rankings, it fully exploits the ordinal information and allows for incomparability. Our method describes the distribution of all partial orders/rankings, avoiding the notorious shortcomings of aggregation. This permits to identify test functions that produce central or outlying rankings of optimizers and to assess the quality of benchmarking suites.

翻译：我们提出了一种根据多个标准在各种测试函数上对优化器进行基准测试的框架。该方法基于最近提出的用于偏序/排序的并集无关通用深度函数，充分利用了序数信息并允许不可比性。我们的方法描述了所有偏序/排序的分布，避免了聚合方法的固有缺陷。这使得我们能够识别产生优化器中心排序或异常排序的测试函数，并评估基准测试集的质量。