Rankings play a crucial role in decision-making. However, if minor changes to items significantly alter their rankings, the quality of the decisions being made can be compromised. The stability of ranking is a measure used to assess how modifications to the ranking algorithm or data affect results. While previous work has focused on stability of the ranking under changes to the algorithm, we introduce a novel measure we refer to as local stability. Local stability indicates the effect of minor changes to the values of an item in the ranking on its rank. Our proposed definition furthermore takes into account the presence of multiple items with similar qualities in the ranking, called dense regions, permitting minor modifications to swap the positions of items within the region. We show that computing this measure in general is hard, and in turn propose a relaxation of the definition to admit approximation. We present (i) LStability, a sampling-based algorithm for approximating local stability, on which we make probably-approximately-correct-type guarantees through the use of concentration inequalities, and (ii) Detect-Dense-Region, an algorithm based on this approach to detect the dense region an item lies in, if it exists. We introduce a number of optimizations to our algorithms to improve their scalability and efficiency. We validate our proposed framework through an extensive suite of experiments, including case studies highlighting the utility of our definitions.
翻译:排名在决策过程中扮演着关键角色。然而,若项目数据的微小变动能显著改变其排名次序,则可能损害决策质量。排名稳定性是用于评估排序算法或数据修改对结果影响程度的度量指标。现有研究主要关注算法变动下的排名稳定性,本文则提出一种称为局部稳定性的新型度量方法。局部稳定性反映了排名中单个项目数值的微小变化对其位次产生的影响。我们提出的定义进一步考虑了排名中存在多个质量相近项目(称为密集区域)的情形,允许在该区域内通过微小调整交换项目位置。我们证明该度量在一般情况下是难以精确计算的,进而提出定义的松弛形式以允许近似求解。我们提出了:(i) LStability——基于采样的局部稳定性近似算法,通过集中不等式给出了概率近似正确性保证;(ii) Detect-Dense-Region——基于此方法检测项目所处密集区域(若存在)的算法。我们引入了多项优化策略以提升算法的可扩展性与效率。通过包含案例研究的综合实验验证了所提框架的有效性,突显了相关定义的实际应用价值。