As the interest in multi- and many-objective optimization algorithms grows, the performance comparison of these algorithms becomes increasingly important. A large number of performance indicators for multi-objective optimization algorithms have been introduced, each of which evaluates these algorithms based on a certain aspect. Therefore, assessing the quality of multi-objective results using multiple indicators is essential to guarantee that the evaluation considers all quality perspectives. This paper proposes a novel multi-metric comparison method to rank the performance of multi-/ many-objective optimization algorithms based on a set of performance indicators. We utilize the Pareto optimality concept (i.e., non-dominated sorting algorithm) to create the rank levels of algorithms by simultaneously considering multiple performance indicators as criteria/objectives. As a result, four different techniques are proposed to rank algorithms based on their contribution at each Pareto level. This method allows researchers to utilize a set of existing/newly developed performance metrics to adequately assess/rank multi-/many-objective algorithms. The proposed methods are scalable and can accommodate in its comprehensive scheme any newly introduced metric. The method was applied to rank 10 competing algorithms in the 2018 CEC competition solving 15 many-objective test problems. The Pareto-optimal ranking was conducted based on 10 well-known multi-objective performance indicators and the results were compared to the final ranks reported by the competition, which were based on the inverted generational distance (IGD) and hypervolume indicator (HV) measures. The techniques suggested in this paper have broad applications in science and engineering, particularly in areas where multiple metrics are used for comparisons. Examples include machine learning and data mining.

翻译：随着对多目标及超多目标优化算法的兴趣日益增长，这些算法的性能比较变得愈发重要。目前已提出了大量用于评估多目标优化算法的性能指标，每个指标都基于特定维度对这些算法进行评价。因此，采用多个指标来评估多目标优化结果的质量至关重要，以确保评估过程涵盖所有质量维度。本文提出一种新颖的多指标比较方法，基于一组性能指标对多目标/超多目标优化算法进行性能排序。我们利用帕累托最优概念（即非支配排序算法），将多个性能指标同时作为准则/目标，构建算法的排序层级。基于此，本文提出了四种不同的技术，依据算法在各帕累托层级的贡献度进行排序。该方法使研究者能够利用一组现有/新开发的性能指标，对多目标/超多目标算法进行充分评估与排序。所提方法具有良好的可扩展性，其综合框架可兼容任何新引入的指标。我们将该方法应用于2018年CEC竞赛中解决15个超多目标测试问题的10个竞争算法排序。基于10个经典多目标性能指标进行帕累托最优排序，并将结果与竞赛基于反向世代距离（IGD）和超体积指标（HV）度量的最终排名进行对比。本文提出的技术在科学与工程领域具有广泛应用前景，特别适用于需要多指标比较的领域，例如机器学习和数据挖掘。