Quantifying uncertainty is critical for the safe deployment of ranking models in real-world applications. Recent work offers a rigorous solution using conformal prediction in a full ranking scenario, which aims to construct prediction sets for the absolute ranks of test items based on the relative ranks of calibration items. However, relying on upper bounds of non-conformity scores renders the method overly conservative, resulting in substantially large prediction sets. To address this, we propose Distribution-informed Conformal Ranking (DCR), which produces efficient prediction sets by deriving the exact distribution of non-conformity scores. In particular, we find that the absolute ranks of calibration items follow Negative Hypergeometric distributions, conditional on their relative ranks. DCR thus uses the rank distribution to derive non-conformity score distribution and determine conformal thresholds. We provide theoretical guarantees that DCR achieves improved efficiency over the baseline while ensuring valid coverage under mild assumptions. Extensive experiments demonstrate the superiority of DCR, reducing average prediction set size by up to 36%, while maintaining valid coverage.

翻译：在现实应用中，量化不确定性对于排序模型的安全部署至关重要。近期研究提出了一种在全排序场景下使用保形预测的严格解决方案，其目标是根据校准项目的相对排序构建测试项目绝对排序的预测集。然而，该方法依赖非保形分数上界的设定导致其过于保守，从而产生过大的预测集。为解决此问题，我们提出基于分布信息的保形排序方法，该方法通过推导非保形分数的精确分布来生成高效的预测集。具体而言，我们发现校准项目的绝对排序在给定其相对排序的条件下服从负超几何分布。因此，DCR利用该排序分布推导非保形分数分布并确定保形阈值。我们提供了理论保证，证明在温和假设下DCR在确保有效覆盖的同时，相比基线方法实现了效率提升。大量实验表明DCR具有显著优越性，在保持有效覆盖的前提下将平均预测集规模降低达36%。