Retrieval-Augmented Generation (RAG) leverages large language models (LLMs) combined with external contexts to enhance the accuracy and reliability of generated responses. However, reliably attributing generated content to specific context segments, context attribution, remains challenging due to the computationally intensive nature of current methods, which often require extensive fine-tuning or human annotation. In this work, we introduce a novel Jensen-Shannon Divergence driven method to Attribute Response to Context (ARC-JSD), enabling efficient and accurate identification of essential context sentences without additional fine-tuning, gradient-calculation or surrogate modelling. Evaluations on a wide range of RAG benchmarks, such as TyDi QA, Hotpot QA, and Musique, using instruction-tuned LLMs in different scales demonstrate superior accuracy and significant computational efficiency improvements compared to the previous surrogate-based method. Furthermore, our mechanistic analysis reveals specific attention heads and multilayer perceptron (MLP) layers responsible for context attribution, providing valuable insights into the internal workings of RAG models and how they affect RAG behaviours. Our code is available at https://github.com/ruizheliUOA/ARC_JSD.

翻译：检索增强生成（RAG）通过结合大型语言模型（LLMs）与外部上下文，提升了生成响应的准确性与可靠性。然而，由于现有方法通常需要大量微调或人工标注，计算成本高昂，将生成内容可靠地归因于特定上下文片段（即上下文归因）仍具挑战性。本研究提出一种新颖的基于Jensen-Shannon散度的方法——上下文响应归因（ARC-JSD），该方法无需额外微调、梯度计算或代理建模，即可高效精准地识别关键上下文句子。通过在TyDi QA、Hotpot QA和Musique等多种RAG基准测试中，使用不同规模的指令微调LLMs进行评估，结果表明：相较于先前的基于代理的方法，本方法在保持卓越准确性的同时，显著提升了计算效率。此外，我们的机制分析揭示了负责上下文归因的特定注意力头和多层感知机（MLP）层，为深入理解RAG模型的内部工作机制及其对RAG行为的影响提供了宝贵见解。代码发布于 https://github.com/ruizheliUOA/ARC_JSD。