Unbiased Learning to Rank (ULTR) aims to train unbiased ranking models from biased click logs, by explicitly modeling a generation process for user behavior and fitting click data based on examination hypothesis. Previous research found empirically that the true latent relevance is mostly recoverable through perfect click fitting. However, we demonstrate that this is not always achievable, resulting in a significant reduction in ranking performance. This research investigates the conditions under which relevance can be recovered from click data at a foundational level. We initially characterize a ranking model as identifiable if it can recover the true relevance up to a scaling transformation, a criterion sufficient for the pairwise ranking objective. Subsequently, we investigate an equivalent condition for identifiability, articulated as a graph connectivity test problem: the recovery of relevance is feasible if and only if the identifiability graph (IG), derived from the underlying structure of the dataset, is connected. The presence of a disconnected IG may lead to degenerate cases and suboptimal ranking performance. To tackle this challenge, we introduce two methods, namely node intervention and node merging, designed to modify the dataset and restore the connectivity of the IG. Empirical results derived from a simulated dataset and two real-world LTR benchmark datasets not only validate our proposed theorems but also demonstrate the effectiveness of our methods in alleviating data bias when the relevance model is unidentifiable.

翻译：无偏学习排序旨在通过明确建模用户行为的生成过程，并基于检验假设拟合点击数据，从有偏的点击日志中训练无偏排序模型。先前研究发现，通过完美的点击拟合，真实潜在相关性大多是可恢复的。然而，我们证明这并非总能实现，导致排序性能显著下降。本研究从基础层面探究从点击数据中恢复相关性的条件。我们首先将排序模型定义为可识别的——若它能恢复至缩放变换下的真实相关性，该准则足以满足成对排序目标。随后，我们研究了可识别性的等价条件，表述为图连通性检验问题：当且仅当从数据集基础结构导出的可识别性图连通时，相关性恢复是可行的。不连通的IG可能导致退化情形与次优排序性能。为应对这一挑战，我们提出节点干预与节点合并两种方法，旨在修改数据集并恢复IG的连通性。基于模拟数据集和两个真实LTR基准数据集的实验结果，不仅验证了我们提出的定理，还证明了这些方法在相关性模型不可识别时缓解数据偏差的有效性。