The rapid progress of research aimed at interpreting the inner workings of advanced language models has highlighted a need for contextualizing the insights gained from years of work in this area. This primer provides a concise technical introduction to the current techniques used to interpret the inner workings of Transformer-based language models, focusing on the generative decoder-only architecture. We conclude by presenting a comprehensive overview of the known internal mechanisms implemented by these models, uncovering connections across popular approaches and active research directions in this area.

翻译：旨在解释先进语言模型内部运作机制的研究进展迅速，这凸显了将多年在该领域获得的见解进行背景化梳理的需求。本入门教材对当前用于解释基于Transformer的语言模型内部运作的技术进行了简洁的技术介绍，重点聚焦于生成式纯解码器架构。最后，我们全面概述了这些模型所实现的已知内部机制，揭示了该领域内流行方法与活跃研究方向之间的联系。