Language models for code such as CodeBERT offer the capability to learn advanced source code representation, but their opacity poses barriers to understanding of captured properties. Recent attention analysis studies provide initial interpretability insights by focusing solely on attention weights rather than considering the wider context modeling of Transformers. This study aims to shed some light on the previously ignored factors of the attention mechanism beyond the attention weights. We conduct an initial empirical study analyzing both attention distributions and transformed representations in CodeBERT. Across two programming languages, Java and Python, we find that the scaled transformation norms of the input better capture syntactic structure compared to attention weights alone. Our analysis reveals characterization of how CodeBERT embeds syntactic code properties. The findings demonstrate the importance of incorporating factors beyond just attention weights for rigorously understanding neural code models. This lays the groundwork for developing more interpretable models and effective uses of attention mechanisms in program analysis.

翻译：诸如CodeBERT等面向代码的语言模型能够学习高级源代码表示，但其不透明性阻碍了对所捕获性质的理解。最近的注意力分析研究仅聚焦于注意力权重，而未考虑Transformer的广泛上下文建模，因而提供了初步的可解释性见解。本研究旨在揭示注意力机制中此前被忽视的（除注意力权重外的）因素。我们开展了一项实证研究，分析CodeBERT中的注意力分布与变换表示。在Java和Python两种编程语言中，我们发现与仅使用注意力权重相比，输入的缩放变换范数能更有效地捕捉语法结构。我们的分析揭示了CodeBERT如何嵌入了代码的语法属性。研究结果表明，在严格理解神经代码模型时，纳入除注意力权重外的因素至关重要。这为开发更具可解释性的模型以及在程序分析中有效运用注意力机制奠定了基础。