Some quality indicators have been proposed for benchmarking preference-based evolutionary multi-objective optimization algorithms using a reference point. Although a systematic review and analysis of the quality indicators are helpful for both benchmarking and practical decision-making, neither has been conducted. In this context, first, this paper reviews existing regions of interest and quality indicators for preference-based evolutionary multi-objective optimization using the reference point. We point out that each quality indicator was designed for a different region of interest. Then, this paper investigates the properties of the quality indicators. We demonstrate that an achievement scalarizing function value is not always consistent with the distance from a solution to the reference point in the objective space. We observe that the regions of interest can be significantly different depending on the position of the reference point and the shape of the Pareto front. We identify undesirable properties of some quality indicators. We also show that the ranking of preference-based evolutionary multi-objective optimization algorithms depends on the choice of quality indicators.

翻译：针对基于参考点的偏好进化多目标优化算法的基准测试，已有多种质量指标被提出。尽管对这些质量指标的系统性综述与分析既有助于基准测试，也有利于实际决策，但此类工作尚未开展。在此背景下，本文首先回顾了现有基于参考点的偏好进化多目标优化中感兴趣的区域及质量指标。我们指出，每种质量指标均针对不同的感兴趣区域而设计。随后，本文研究了这些质量指标的特性。我们证明，成就标量化函数值在目标空间中与解到参考点距离的一致性并非总是成立。我们观察到，感兴趣区域可能因参考点位置及帕累托前沿形状的不同而显著变化。我们识别出部分质量指标存在不良特性。同时表明，偏好进化多目标优化算法的排序结果取决于所选取的质量指标。