Retrieval-augmented generation systems for legal question answering typically retrieve passages based on semantic similarity and provide them to a language model, which then generates cited answers. Prior work assumes that highly ranked passages are most likely to be usefully cited by the model. Perturbation-based attribution methods, such as C-LIME, have been used exclusively for post-hoc explanation. However, on the AQuAECHR benchmark, semantic similarity does not correlate with passage attribution. Within a retriever's candidate pool, similarity-based ranking performs worse than random selection at surfacing gold citation paragraphs. To address this limitation, a lightweight cross-encoder is trained on continuous perturbation-based attribution scores to re-rank passages prior to generation. This approach is evaluated on the AQuAECHR benchmark, using two language models and five-fold cross-validation. The re-ranker substantially improves citation faithfulness and alignment with gold expert answers. Notably, two re-rankers trained independently on different models converge beyond their raw attribution agreement. This finding indicates that the cross-encoder reduces model-specific noise and produces a shared relevance signal that partially transfers across models, although same-model re-ranking remains more effective. These results demonstrate that perturbation-based attribution provides a practical, model-agnostic training signal for citation-aware retrieval.
翻译:摘要:面向法律问答的检索增强生成系统通常基于语义相似度检索段落并输入语言模型,由其生成带引用的答案。既有研究假设高排序段落最有可能被模型有效引用。扰动归因方法(如C-LIME)此前仅用于事后解释。然而,在AQuAECHR基准测试中,语义相似度与段落归因并不相关。在检索器的候选池中,基于相似度的排序在呈现黄金引用段落方面的表现甚至不如随机选择。为解决此局限,本文训练了一个轻量级跨编码器,基于连续扰动归因得分在生成前对段落进行重排序。该方法在AQuAECHR基准上使用两个语言模型及五折交叉验证进行评估。重排序器显著提升了引用忠实度及与专家黄金答案的对齐程度。值得注意的是,基于不同模型独立训练的两个重排序器展现的收敛性超越了其原始归因一致性。这一发现表明,跨编码器可降低模型特定噪声,产生可跨模型部分迁移的共享相关性信号,尽管同模型重排序仍更有效。这些结果证明,扰动归因能为引用感知检索提供实用的、模型无关的训练信号。