Transformer networks have revolutionized NLP representation learning since they were introduced. Though a great effort has been made to explain the representation in transformers, it is widely recognized that our understanding is not sufficient. One important reason is that there lack enough visualization tools for detailed analysis. In this paper, we propose to use dictionary learning to open up these "black boxes" as linear superpositions of transformer factors. Through visualization, we demonstrate the hierarchical semantic structures captured by the transformer factors, e.g., word-level polysemy disambiguation, sentence-level pattern formation, and long-range dependency. While some of these patterns confirm the conventional prior linguistic knowledge, the rest are relatively unexpected, which may provide new insights. We hope this visualization tool can bring further knowledge and a better understanding of how transformer networks work. The code is available at https://github.com/zeyuyun1/TransformerVis

翻译：Transformer网络自提出以来彻底革新了自然语言处理领域的表征学习。尽管已有大量工作致力于解释Transformer中的表征机制，但业界普遍认为现有理解仍不充分。一个重要原因是缺乏足够的可视化工具进行细致分析。本文提出采用词典学习的方法，将这些"黑箱"解构为Transformer因子的线性叠加。通过可视化，我们展示了Transformer因子捕获的分层语义结构，例如：词级多义词消歧、句级模式构建以及长距离依存关系。其中部分模式印证了传统先验语言学知识，其余则为相对意外的发现，可能提供新的见解。希望本可视化工具能促进对Transformer网络工作机制的深入认知。代码已开源至https://github.com/zeyuyun1/TransformerVis