We report the presence of a simple neural mechanism that represents an input-output function as a vector within autoregressive transformer language models (LMs). Using causal mediation analysis on a diverse range of in-context-learning (ICL) tasks, we find that a small number attention heads transport a compact representation of the demonstrated task, which we call a function vector (FV). FVs are robust to changes in context, i.e., they trigger execution of the task on inputs such as zero-shot and natural text settings that do not resemble the ICL contexts from which they are collected. We test FVs across a range of tasks, models, and layers and find strong causal effects across settings in middle layers. We investigate the internal structure of FVs and find while that they often contain information that encodes the output space of the function, this information alone is not sufficient to reconstruct an FV. Finally, we test semantic vector composition in FVs, and find that to some extent they can be summed to create vectors that trigger new complex tasks. Our findings show that compact, causal internal vector representations of function abstractions can be explicitly extracted from LLMs. Our code and data are available at https://functions.baulab.info.

翻译：我们报告了自回归Transformer语言模型（LMs）中存在一种简单的神经机制，该机制将输入-输出函数表示为向量。通过对多种上下文学习（ICL）任务进行因果中介分析，我们发现少数注意力头会传输所演示任务的紧凑表示，我们将其称为函数向量（FV）。函数向量对上下文变化具有鲁棒性，即它们能在零样本和自然文本等与ICL上下文不同的输入设置中触发任务的执行。我们在多种任务、模型和层级上测试了函数向量，发现中间层在各类设置中均表现出显著的因果效应。我们研究了函数向量的内部结构，发现尽管它们通常包含编码函数输出空间的信息，但仅凭这些信息不足以重构函数向量。最后，我们测试了函数向量中的语义向量组合，发现它们在一定程度上可以通过求和来创建触发新复杂任务的向量。我们的研究结果表明，可以从LLMs中显式提取函数抽象的紧凑、因果性内部向量表示。我们的代码和数据可在 https://functions.baulab.info 获取。