Large language models (LLMs) exhibit remarkable performance improvement through in-context learning (ICL) by leveraging task-specific examples in the input. However, the mechanisms behind this improvement remain elusive. In this work, we investigate how LLM embeddings and attention representations change following in-context-learning, and how these changes mediate improvement in behavior. We employ neuroscience-inspired techniques such as representational similarity analysis (RSA) and propose novel methods for parameterized probing and measuring ratio of attention to relevant vs. irrelevant information in Llama-2 70B and Vicuna 13B. We designed two tasks with a priori relationships among their conditions: linear regression and reading comprehension. We formed hypotheses about expected similarities in task representations and measured hypothesis alignment of LLM representations before and after ICL as well as changes in attention. Our analyses revealed a meaningful correlation between improvements in behavior after ICL and changes in both embeddings and attention weights across LLM layers. This empirical framework empowers a nuanced understanding of how latent representations shape LLM behavior, offering valuable tools and insights for future research and practical applications.

翻译：大语言模型（LLMs）通过利用输入中的任务特定示例进行上下文学习（ICL），展现出显著的性能提升。然而，这一改进背后的机制仍然难以捉摸。本研究探讨了LLM的嵌入和注意力表征在上下文学习后如何变化，以及这些变化如何介导行为改进。我们采用受神经科学启发的技术，如表征相似性分析（RSA），并提出了参数化探针以及测量Llama-2 70B和Vicuna 13B中注意力与相关及无关信息比例的新方法。我们设计了两个任务，其条件之间存在先验关系：线性回归和阅读理解。我们形成了关于任务表征预期相似性的假设，并测量了ICL前后LLM表征的假设对齐程度以及注意力的变化。我们的分析揭示了ICL后行为改进与LLM各层中嵌入和注意力权重变化之间存在有意义的相关性。这一经验框架增强了对潜在表征如何塑造LLM行为的细致理解，为未来研究和实际应用提供了宝贵的工具和见解。