A growing body of research has demonstrated that the behavior of large language models can be effectively controlled at inference time by directly modifying their internal states, either through vector additions to their activations or through updates to their weight matrices. These techniques, while powerful, are often guided by empirical heuristics, such as deriving steering vectors from the average activations of contrastive prompts. This work provides a theoretical foundation for these interventions, explaining how they emerge from the fundamental computations of the transformer architecture. Building on the recent finding that a prompt's influence can be mathematically mapped to token-dependent implicit weight updates (Dherin et. al, 2025), we derive a principled method for condensing this information into token-independent thought vectors and thought matrices. These constructs provide a theoretical explanation for existing vector-and-matrix-based model editing techniques and offer a direct, computationally-grounded method for transmuting textual input into reusable weight updates.

翻译：越来越多的研究表明，通过直接修改大型语言模型的内部状态——无论是通过对其激活值进行向量加法，还是通过更新其权重矩阵——可以在推理时有效地控制其行为。这些技术虽然强大，但通常依赖于经验启发式方法，例如从对比性提示的平均激活值中推导出引导向量。本研究为这些干预措施提供了理论基础，解释了它们如何从Transformer架构的基本计算中产生。基于最近的研究发现（Dherin等人，2025），即提示的影响可以通过数学方式映射为依赖于标记的隐式权重更新，我们推导出一种原则性方法，将此信息压缩为不依赖于标记的思维向量和思维矩阵。这些构造为现有的基于向量和矩阵的模型编辑技术提供了理论解释，并提供了一种直接的、基于计算的将文本输入转化为可重用权重更新的方法。