Large Language Models (LLMs) for code are a family of high-parameter, transformer-based neural networks pre-trained on massive datasets of both natural and programming languages. These models are rapidly being employed in commercial AI-based developer tools, such as GitHub CoPilot. However, measuring and explaining their effectiveness on programming tasks is a challenging proposition, given their size and complexity. The methods for evaluating and explaining LLMs for code are inextricably linked. That is, in order to explain a model's predictions, they must be reliably mapped to fine-grained, understandable concepts. Once this mapping is achieved, new methods for detailed model evaluations are possible. However, most current explainability techniques and evaluation benchmarks focus on model robustness or individual task performance, as opposed to interpreting model predictions. To this end, this paper introduces ASTxplainer, an explainability method specific to LLMs for code that enables both new methods for LLM evaluation and visualizations of LLM predictions that aid end-users in understanding model predictions. At its core, ASTxplainer provides an automated method for aligning token predictions with AST nodes, by extracting and aggregating normalized model logits within AST structures. To demonstrate the practical benefit of ASTxplainer, we illustrate the insights that our framework can provide by performing an empirical evaluation on 12 popular LLMs for code using a curated dataset of the most popular GitHub projects. Additionally, we perform a user study examining the usefulness of an ASTxplainer-derived visualization of model predictions aimed at enabling model users to explain predictions. The results of these studies illustrate the potential for ASTxplainer to provide insights into LLM effectiveness, and aid end-users in understanding predictions.

翻译：面向代码的大型语言模型（LLMs）是一类基于Transformer架构的高参数神经网络，通过在自然语言和编程语言的海量数据集上进行预训练而得到。这类模型正迅速被应用于商业化的AI开发工具中，例如GitHub Copilot。然而，鉴于其规模与复杂性，衡量并解释它们在编程任务中的有效性是一项具有挑战性的课题。评估与解释面向代码的LLMs的方法密不可分——要解释模型的预测，必须将其可靠地映射到细粒度、可理解的概念上；一旦实现这种映射，便可催生出新的精细模型评估方法。然而，当前大多数可解释性技术与评估基准侧重于模型鲁棒性或单一任务性能，而非解释模型预测。为此，本文提出ASTxplainer——一种专为面向代码的LLMs设计的可解释性方法，它既能支持新的LLM评估手段，又能生成有助于终端用户理解模型预测的可视化结果。ASTxplainer的核心机制是通过提取并聚合AST结构内归一化的模型logits，提供一种自动化的方法将词元预测与AST节点对齐。为展示ASTxplainer的实用价值，我们利用一个基于热门GitHub项目精选数据集，对12种主流面向代码的LLMs进行了实证评估，揭示了该框架能提供的深层洞察。此外，我们开展了一项用户研究，考察基于ASTxplainer生成的模型预测可视化是否有助于模型用户解释预测结果。这些研究表明，ASTxplainer具有揭示LLM有效性并为终端用户理解预测提供助力的潜力。