Understanding Transformer-based models has attracted significant attention, as they lie at the heart of recent technological advances across machine learning. While most interpretability methods rely on running models over inputs, recent work has shown that a zero-pass approach, where parameters are interpreted directly without a forward/backward pass is feasible for some Transformer parameters, and for two-layer attention networks. In this work, we present a theoretical analysis where all parameters of a trained Transformer are interpreted by projecting them into the embedding space, that is, the space of vocabulary items they operate on. We derive a simple theoretical framework to support our arguments and provide ample evidence for its validity. First, an empirical analysis showing that parameters of both pretrained and fine-tuned models can be interpreted in embedding space. Second, we present two applications of our framework: (a) aligning the parameters of different models that share a vocabulary, and (b) constructing a classifier without training by ``translating'' the parameters of a fine-tuned classifier to parameters of a different model that was only pretrained. Overall, our findings open the door to interpretation methods that, at least in part, abstract away from model specifics and operate in the embedding space only.

翻译：理解基于Transformer的模型已引起广泛关注，因为这些模型是近年来机器学习领域多项技术突破的核心。虽然大多数可解释性方法依赖于在输入数据上运行模型，但近期研究表明，一种“零前向/后向传播”方法——即直接解释参数而无需前向/后向传播——对于某些Transformer参数及两层注意力网络是可行的。在本工作中，我们提出一种理论分析，通过将所有训练后的Transformer参数投影到嵌入空间（即其所处理的词汇项所在的空间）中进行解释。我们推导出一个简单的理论框架以支撑我们的论点，并提供充分证据证明其有效性。首先，通过实证分析表明，预训练模型和微调模型的参数均可在嵌入空间中得到解释。其次，我们展示了该框架的两项应用：（a）对齐共享词汇表的不同模型参数；（b）通过将微调分类器的参数“翻译”为仅经过预训练的不同模型参数，在不经训练的情况下构建分类器。总体而言，我们的发现为可解释性方法开辟了新途径——这些方法至少能部分脱离具体模型的细节，仅需在嵌入空间中运作。