Language Models (LMs) have shown their application for tasks pertinent to code and several code~LMs have been proposed recently. The majority of the studies in this direction only focus on the improvements in performance of the LMs on different benchmarks, whereas LMs are considered black boxes. Besides this, a handful of works attempt to understand the role of attention layers in the code~LMs. Nonetheless, feed-forward layers remain under-explored which consist of two-thirds of a typical transformer model's parameters. In this work, we attempt to gain insights into the inner workings of code language models by examining the feed-forward layers. To conduct our investigations, we use two state-of-the-art code~LMs, Codegen-Mono and Ploycoder, and three widely used programming languages, Java, Go, and Python. We focus on examining the organization of stored concepts, the editability of these concepts, and the roles of different layers and input context size variations for output generation. Our empirical findings demonstrate that lower layers capture syntactic patterns while higher layers encode abstract concepts and semantics. We show concepts of interest can be edited within feed-forward layers without compromising code~LM performance. Additionally, we observe initial layers serve as ``thinking'' layers, while later layers are crucial for predicting subsequent code tokens. Furthermore, we discover earlier layers can accurately predict smaller contexts, but larger contexts need critical later layers' contributions. We anticipate these findings will facilitate better understanding, debugging, and testing of code~LMs.

翻译：语言模型（LMs）在代码相关任务中已展现出应用潜力，近期亦涌现出若干代码~LMs。该领域多数研究仅关注语言模型在不同基准测试中的性能提升，而将语言模型视为黑箱系统。此外，少量研究尝试探究注意力层在代码~LMs中的作用，但前馈层——其参数量占典型Transformer模型总参数的三分之二——仍未得到充分探索。本研究通过剖析前馈层，尝试揭示代码语言模型的内在运作机制。我们采用两种先进代码~LMs（Codegen-Mono与Ploycoder）及三种广泛使用的编程语言（Java、Go、Python）展开实验，重点探究存储概念的组织方式、概念的可编辑性，以及不同层级与输入上下文规模变化对输出生成的影响。实证结果表明：底层网络主要捕获句法模式，而高层网络则编码抽象概念与语义信息；我们证明可在前馈层中编辑目标概念而不影响代码~LM性能；此外，初始层承担"思考"功能，后续层对预测后续代码标记至关重要；我们还发现较早层能准确预测较小上下文，但较大上下文需要关键后续层的协同处理。这些发现有望促进对代码~LMs的理解、调试与测试工作。