Some quality indicators have been proposed for benchmarking preference-based evolutionary multi-objective optimization algorithms using a reference point. Although a systematic review and analysis of the quality indicators are helpful for both benchmarking and practical decision-making, neither has been conducted. In this context, first, this paper reviews existing regions of interest and quality indicators for preference-based evolutionary multi-objective optimization using the reference point. We point out that each quality indicator was designed for a different region of interest. Then, this paper investigates the properties of the quality indicators. We demonstrate that an achievement scalarizing function value is not always consistent with the distance from a solution to the reference point in the objective space. We observe that the regions of interest can be significantly different depending on the position of the reference point and the shape of the Pareto front. We identify undesirable properties of some quality indicators. We also show that the ranking of preference-based evolutionary multi-objective optimization algorithms depends on the choice of quality indicators.

翻译：针对使用参考点的偏好进化多目标优化算法，已有若干质量指标被提出用于性能基准测试。尽管系统性地综述与分析这些质量指标既有助于基准测试实践，也有利于实际决策，但相关工作尚属空白。为此，本文首先回顾了现有基于参考点的偏好进化多目标优化中定义的关注区域与质量指标，指出每个质量指标均针对不同的关注区域设计。继而，本文探究了这些质量指标的属性，证明成就标量化函数值并非始终与目标空间中解到参考点的距离一致，并观察到关注区域会因参考点位置及Pareto前沿形状的不同而产生显著差异。本文识别出部分质量指标的不良特性，同时揭示偏好进化多目标优化算法的排序结果依赖于质量指标的选择。