Ranking systems influence decision-making in high-stakes domains like health, education, and employment, where they can have substantial economic and social impacts. This makes the integration of safety mechanisms essential. One such mechanism is abstention, which enables algorithmic decision-making systems to defer uncertain or low-confidence decisions to human experts. While abstention has been predominantly explored in the context of classification tasks, its application to other machine learning paradigms remains underexplored. In this paper, we introduce a novel method for abstention in pairwise learning-to-rank tasks. Our approach is based on thresholding the ranker's conditional risk: the system abstains from making a decision when the estimated risk exceeds a predefined threshold. Our contributions are threefold: a theoretical characterization of the optimal abstention strategy, a model-agnostic, plug-in algorithm for constructing abstaining ranking models, and a comprehensive empirical evaluation across multiple datasets, demonstrating the effectiveness of our approach.

翻译：排序系统在健康、教育和就业等高风险领域影响决策，可能产生重大的经济和社会影响，因此整合安全机制至关重要。其中一种机制是弃权，它使算法决策系统能够将不确定或低置信度的决策交由人类专家处理。虽然弃权主要在分类任务中得到探索，但其在机器学习其他范式中的应用仍研究不足。本文针对对级学习排序任务提出了一种新的弃权方法。我们的方法基于对排序器条件风险的阈值化：当估计风险超过预设阈值时，系统弃权不做决策。我们的贡献有三方面：最优弃权策略的理论刻画、一种模型无关的即插式构建弃权排序模型的算法，以及跨多个数据集的综合实证评估，证明了我们方法的有效性。