Programmers often struggle to identify and fix bugs in their programs. In recent years, many language models (LMs) have been proposed to fix erroneous programs and support error recovery. However, the LMs tend to generate solutions that differ from the original input programs. This leads to potential comprehension difficulties for users. In this paper, we propose an approach to suggest a correct program with minimal repair edits using CodeT5. We fine-tune a pre-trained CodeT5 on code pairs of wrong and correct programs and evaluate its performance with several baseline models. The experimental results show that the fine-tuned CodeT5 achieves a pass@100 of 91.95% and an average edit distance of the most similar correct program of 6.84, which indicates that at least one correct program can be suggested by generating 100 candidate programs. We demonstrate the effectiveness of LMs in suggesting program repair with minimal edits for solving introductory programming problems.
翻译:程序员常常难以识别并修复其程序中的错误。近年来,众多语言模型被提出用于修正错误程序并支持错误恢复。然而,这些语言模型往往生成的解决方案与原始输入程序存在差异,这可能导致用户理解困难。本文提出一种方法,利用CodeT5通过最小修复编辑来推荐正确程序。我们针对错误与正确程序组成的代码对,对预训练的CodeT5进行微调,并与多个基准模型评估其性能。实验结果表明,微调后的CodeT5实现了91.95%的pass@100值,且最相似正确程序的平均编辑距离为6.84,这意味着通过生成100个候选程序可推荐至少一个正确程序。我们证实了语言模型在通过最小编辑建议程序修复以解决入门级编程问题方面的有效性。