Programming language understanding and representation (a.k.a code representation learning) has always been a hot and challenging task in software engineering. It aims to apply deep learning techniques to produce numerical representations of the source code features while preserving its semantics. These representations can be used for facilitating subsequent code-related tasks. The abstract syntax tree (AST), a fundamental code feature, illustrates the syntactic information of the source code and has been widely used in code representation learning. However, there is still a lack of systematic and quantitative evaluation of how well AST-based code representation facilitates subsequent code-related tasks. In this paper, we first conduct a comprehensive empirical study to explore the effectiveness of the AST-based code representation in facilitating follow-up code-related tasks. To do so, we compare the performance of models trained with code token sequence (Token for short) based code representation and AST-based code representation on three popular types of code-related tasks. Surprisingly, the overall quantitative statistical results demonstrate that models trained with AST-based code representation consistently perform worse across all three tasks compared to models trained with Token-based code representation. Our further quantitative analysis reveals that models trained with AST-based code representation outperform models trained with Token-based code representation in certain subsets of samples across all three tasks. We also conduct comprehensive experiments to evaluate and reveal the impact of the choice of AST parsing/preprocessing/encoding methods on AST-based code representation and subsequent code-related tasks. Our study provides future researchers with detailed guidance on how to select solutions at each stage to fully exploit AST.

翻译：编程语言理解与表示（亦称代码表示学习）一直是软件工程领域富有挑战性的热点任务。其目标在于运用深度学习技术，在保留源代码语义的同时生成其数值化表示，这些表示可为后续代码相关任务提供支撑。抽象语法树（AST）作为源代码的基础特征，揭示了源码的语法信息，已被广泛用于代码表示学习。然而，关于AST驱动的代码表示对后续代码相关任务的促进效果，仍缺乏系统性量化评估。本文首先开展全面的实证研究，探究AST驱动的代码表示对后续代码相关任务的实际效用。为此，我们基于代码标识符序列（简称Token）表示与AST表示训练的模型，在三种典型代码相关任务上进行了性能对比。令人惊讶的是，整体量化统计结果表明：在三项任务中，基于AST表示训练的模型性能均逊于基于Token表示训练的模型。进一步量化分析揭示，虽然整体表现不佳，但在三项任务的特定样本子集中，基于AST表示训练的模型仍能超越Token表示训练的模型。我们还通过全面实验，评估并揭示了AST解析/预处理/编码方法的选择对AST表示及其后续代码相关任务的影响。本研究为未来研究者提供了各环节策略选择的详细指南，助力充分挖掘AST的潜力。