Recent studies show that neural retrievers often display source bias, favoring passages generated by LLMs over human-written ones, even when both are semantically similar. This bias has been considered an inherent flaw of retrievers, raising concerns about the fairness and reliability of modern information access systems. Our work challenges this view by showing that source bias stems from supervision in retrieval datasets rather than the models themselves. We found that non-semantic differences, like fluency and term specificity, exist between positive and negative documents, mirroring differences between LLM and human texts. In the embedding space, the bias direction from negatives to positives aligns with the direction from human-written to LLM-generated texts. We theoretically show that retrievers inevitably absorb the artifact imbalances in the training data during contrastive learning, which leads to their preferences over LLM texts. To mitigate the effect, we propose two approaches: 1) reducing artifact differences in training data and 2) adjusting LLM text vectors by removing their projection on the bias vector. Both methods substantially reduce source bias. We hope our study alleviates some concerns regarding LLM-generated texts in information access systems.

翻译：近期研究表明，神经检索器常表现出来源偏差——即使语义相似，其也更倾向于选择由大语言模型生成的段落而非人工撰写的段落。这种偏差被视为检索器的固有缺陷，引发了对现代信息访问系统公平性与可靠性的担忧。本研究通过证明来源偏差源于检索数据集的监督信号而非模型本身，对此观点提出质疑。我们发现，正负文档之间存在非语义差异（如流畅性与术语特异性），这与LLM文本与人类文本的差异特征相呼应。在嵌入空间中，从负样本到正样本的偏差方向与从人类文本到LLM生成文本的方向一致。我们理论证明，对比学习过程中检索器必然吸收训练数据中的伪影失衡，从而导致其对LLM文本的偏好。为缓解该效应，我们提出两种方法：1）减少训练数据中的伪影差异；2）通过移除LLM文本向量在偏差方向上的投影来调整向量。两种方法均显著降低来源偏差。我们希望本研究能缓解对信息访问系统中LLM生成文本的部分担忧。