Previous studies have found that PLM-based retrieval models exhibit a preference for LLM-generated content, assigning higher relevance scores to these documents even when their semantic quality is comparable to human-written ones. This phenomenon, known as source bias, threatens the sustainable development of the information access ecosystem. However, the underlying causes of source bias remain unexplored. In this paper, we explain the process of information retrieval with a causal graph and discover that PLM-based retrievers learn perplexity features for relevance estimation, causing source bias by ranking the documents with low perplexity higher. Theoretical analysis further reveals that the phenomenon stems from the positive correlation between the gradients of the loss functions in language modeling task and retrieval task. Based on the analysis, a causal-inspired inference-time debiasing method is proposed, called Causal Diagnosis and Correction (CDC). CDC first diagnoses the bias effect of the perplexity and then separates the bias effect from the overall estimated relevance score. Experimental results across three domains demonstrate the superior debiasing effectiveness of CDC, emphasizing the validity of our proposed explanatory framework. Source codes are available at https://github.com/WhyDwelledOnAi/Perplexity-Trap.

翻译：先前研究发现，基于预训练语言模型的检索模型对大型语言模型生成的内容存在偏好，即使其语义质量与人类撰写文档相当，仍会赋予更高的相关性分数。这种现象被称为来源偏差，威胁着信息获取生态系统的可持续发展。然而，来源偏差的根本原因尚未得到充分探究。本文通过因果图解释信息检索过程，发现基于PLM的检索模型会学习困惑度特征进行相关性估计，从而因对低困惑度文档给予更高排序而导致来源偏差。理论分析进一步揭示，该现象源于语言建模任务与检索任务中损失函数梯度之间的正相关性。基于此分析，我们提出一种因果启发的推理时去偏方法，称为因果诊断与校正。该方法首先诊断困惑度的偏差效应，随后将偏差效应从整体估计的相关性分数中分离。在三个领域内的实验结果表明，因果诊断与校正具有优越的去偏效果，验证了我们所提解释框架的有效性。源代码发布于https://github.com/WhyDwelledOnAi/Perplexity-Trap。