Retrieval-augmented generation (RAG) enhances large language models (LLMs) by conditioning generation on retrieved external documents, but the effect of retrieved context is often non-trivial. In realistic retrieval settings, the retrieved document set often contains a mixture of documents that vary in relevance and usefulness. While prior work has largely examined these phenomena through output behavior, little is known about how retrieved context shapes the internal representations that mediate information integration in RAG. In this work, we study RAG through the lens of latent representations. We systematically analyze how different types of retrieved documents affect the hidden states of LLMs, and how these internal representation shifts relate to downstream generation behavior. Across four question-answering datasets and three LLMs, we analyze internal representations under controlled single- and multi-document settings. Our results reveal how context relevancy and layer-wise processing influence internal representations, providing explanations on LLMs output behaviors and insights for RAG system design.

翻译：检索增强生成（RAG）通过基于检索到的外部文档来调节生成过程，从而增强大语言模型（LLMs）的性能，但检索上下文的影响通常并非微不足道。在实际的检索场景中，检索到的文档集通常包含相关性和实用性各不相同的混合文档。尽管先前的研究主要通过输出行为来考察这些现象，但对于检索上下文如何塑造RAG中信息整合所依赖的内部表征，目前知之甚少。在本工作中，我们从潜在表征的视角研究RAG。我们系统分析了不同类型的检索文档如何影响LLMs的隐藏状态，以及这些内部表征的转变如何与下游生成行为相关联。通过在四个问答数据集和三种LLMs上，对受控的单文档和多文档设置下的内部表征进行分析，我们的结果揭示了上下文相关性和逐层处理如何影响内部表征，从而为解释LLMs的输出行为以及为RAG系统设计提供见解。