A major problem in the study of large language models is to understand their inherent low-dimensional structure. We introduce an approach to study the low-dimensional structure of language models at a model-agnostic level: as sequential probabilistic models. We first empirically demonstrate that a wide range of modern language models exhibit low-rank structure: in particular, matrices built from the model's logits for varying sets of prompts and responses have low approximate rank. We then show that this low-rank structure can be leveraged for generation -- in particular, we can generate a response to a target prompt using a linear combination of the model's outputs on unrelated, or even nonsensical prompts. On the theoretical front, we observe that studying the approximate rank of language models in the sense discussed above yields a simple universal abstraction whose theoretical predictions parallel our experiments. We then analyze the representation power of the abstraction and give provable learning guarantees.

翻译：大型语言模型研究中的一个核心问题是理解其固有的低维结构。我们提出了一种在模型无关层面研究语言模型低维结构的方法：将其视为序列概率模型。首先，我们通过实验证明，一系列现代语言模型均表现出低秩结构：具体而言，由模型在不同提示集和响应集上的逻辑值构建的矩阵具有较低的近似秩。随后，我们展示了如何利用这种低秩结构进行生成——特别是，我们可以通过线性组合模型在无关甚至无意义提示上的输出来生成对目标提示的响应。在理论层面，我们指出，通过上述近似秩的角度研究语言模型，可得到一个简单的通用抽象框架，其理论预测与我们的实验结果相吻合。我们进一步分析了该抽象框架的表征能力，并给出了可证明的学习保证。