Retrieval-augmented generation (RAG) systems can respond incorrectly even when the correct passage was retrieved. The model must still read the retrieved passages and identify which one contains the answer among others that look relevant. This passage-reading model is called the reader. Does it fail simply because the context is longer or because the other passages genuinely compete with the correct one? We introduce and demonstrate a matched-control protocol for RAG reading: we keep the number and length of passages fixed, but replace hard competitors with less competitive real passages. We apply this control across two compact open models on SQuAD. This replacement partially restores performance, with the strongest effects on F1 and answer inclusion. For Phi-2, this recovers +6.0 EM points, +7.0 answer-inclusion points, and +0.057 F1. For Qwen2.5-1.5B, it recovers +4.5 EM points, +9.0 answer-inclusion points, and +0.068 F1. To track how performance changes as competitors accumulate, we also report retention curves and summarize them with a right-censored half-life when the curves do not cross half-retention. Together, these results show the protocol isolates a competition effect distinct from context length, though the effect is clearer for F1 and answer inclusion than for exact match, and also varies with snippet length.
翻译:检索增强生成(RAG)系统即使在检索到正确段落时仍可能给出错误响应。模型必须阅读检索到的段落,并从看似相关的段落中识别出包含答案的正确段落。这种段落阅读模型被称为阅读器。其失败原因究竟是上下文过长,还是其他段落与正确段落之间存在实质性竞争?我们引入并论证了一种用于RAG阅读的匹配对照协议:固定段落数量与长度不变,但将强竞争段落替换为竞争性较弱的真实段落。我们在两个紧凑型开源模型上基于SQuAD数据集实施该对照实验。这种替换部分恢复了模型性能,其中对F1分数和答案包含率的影响最为显著。对于Phi-2模型,该方法恢复了+6.0的精确匹配(EM)分数、+7.0的答案包含率以及+0.057的F1分数。对于Qwen2.5-1.5B模型,则恢复了+4.5的EM分数、+9.0的答案包含率以及+0.068的F1分数。为追踪模型性能随竞争段落积累的变化趋势,我们进一步绘制了保留曲线,并在曲线未达到半保留阈值时采用右删失半衰期进行总结。综合结果表明,该协议能够分离出独立于上下文长度的竞争效应——尽管该效应对F1分数和答案包含率的影响比精确匹配更显著,且其强度随片段长度变化而不同。