Analyses of transformer-based models have shown that they encode a variety of linguistic information from their textual input. While these analyses have shed a light on the relation between linguistic information on one side, and internal architecture and parameters on the other, a question remains unanswered: how is this linguistic information reflected in sentence embeddings? Using datasets consisting of sentences with known structure, we test to what degree information about chunks (in particular noun, verb or prepositional phrases), such as grammatical number, or semantic role, can be localized in sentence embeddings. Our results show that such information is not distributed over the entire sentence embedding, but rather it is encoded in specific regions. Understanding how the information from an input text is compressed into sentence embeddings helps understand current transformer models and help build future explainable neural models.

翻译：对基于Transformer的模型分析表明，这些模型能够从文本输入中编码多种语言信息。尽管现有分析揭示了语言信息与内部架构及参数之间的关系，但一个关键问题仍未得到解答：这些语言信息如何在句子嵌入中得到体现？本研究使用具有已知结构的句子数据集，系统检验了关于语块（特别是名词短语、动词短语或介词短语）的信息（如语法数或语义角色）在句子嵌入中的可定位程度。实验结果表明，此类信息并非均匀分布于整个句子嵌入中，而是编码于特定区域。理解输入文本信息如何被压缩至句子嵌入中，不仅有助于解析当前Transformer模型的工作机制，也将为构建未来可解释的神经模型提供理论基础。