Automated program repair (APR) aims to fix software bugs automatically without human debugging efforts and plays a crucial role in software development and maintenance. Despite the recent significant progress, APR is still challenged by a long-standing overfitting problem (i.e., the generated patch is plausible but overfitting). Various techniques have thus been proposed to address the overfitting problem. Among them, leveraging deep learning approaches to predict patch correctness is emerging along with the available large-scale patch benchmarks recently. However, existing learning-based techniques mainly rely on manually-designed code features, which can be extremely costly and challenging to construct in practice. In this paper, we propose APPT, a pre-trained model-based automated patch correctness assessment technique, which treats the source code as token sequences without extra overhead to design hand-crafted features. In particular, APPT adopts a pre-trained model as the encoder stack, followed by an LSTM stack and a deep learning classifier. Although our idea is general and can be built on various pre-trained models, we implemente APPT based on the BERT model. We conduct an extensive experiment on 1,183 Defects4J patches and the results show that APPT achieves prediction accuracy of 79.0% and recall of 81.3%, outperforming the state-of-the-art technique CACHE by 3.6% and 4.8%. Our additional investigation on 49,694 real-world patches shows that APPT achieves the optimum performance (exceeding 99% in five common metrics for assessing patch classification techniques) compared with existing representation learning techniques. We also prove that adopting code pre-trained models can further provide substantial advancement (e.g., GraphCodeBERT-based APPT improves BERT-based APPT by 3.0% and 2.6% in precision and recall, respectively), highlighting the generalizability of APPT.

翻译：自动化程序修复（APR）旨在无需人工调试即可自动修复软件缺陷，在软件开发和维护中具有关键作用。尽管近年来取得了显著进展，APR仍面临长期存在的过拟合问题（即生成的补丁看似合理实则过拟合）。为此，研究者提出了多种技术来应对过拟合问题。其中，随着近期大规模补丁基准数据的涌现，利用深度学习方法预测补丁正确性逐渐成为新兴方向。然而，现有基于学习的技术主要依赖人工设计的代码特征，这在实践中构建成本极高且充满挑战。本文提出APPT——一种基于预训练模型的自动化补丁正确性评估技术，该方法将源代码视为标记序列，无需额外开销设计手工特征。具体而言，APPT采用预训练模型作为编码器堆栈，其后接LSTM堆栈与深度学习分类器。尽管该思想具有通用性且可基于多种预训练模型构建，我们基于BERT模型实现了APPT。在涵盖1183个Defects4J补丁的广泛实验中，APPT实现了79.0%的预测准确率与81.3%的召回率，较当前最先进的CACHE技术分别提升3.6%和4.8%。对49694个真实世界补丁的进一步研究表明，相比现有表示学习技术，APPT在五个常用补丁分类评估指标中均达到最优性能（超过99%）。我们还证明，采用代码预训练模型可带来显著性能提升（例如，基于GraphCodeBERT的APPT在精确率和召回率上较基于BERT的APPT分别提升3.0%和2.6%），充分体现了APPT的泛化能力。