Automated Program Repair (APR) has evolved significantly with the advent of Large Language Models (LLMs). Fine-tuning LLMs for program repair is a recent avenue of research, with many dimensions which have not been explored. Existing work mostly fine-tunes LLMs with naive code representations and is fundamentally limited in its ability to fine-tune larger LLMs. To address this problem, we propose RepairLLaMA, a novel program repair approach that combines 1) code representations for APR and 2) the state-of-the-art parameter-efficient LLM fine-tuning technique called LoRA. This results in RepairLLaMA producing a highly effective `program repair adapter' for fixing bugs with language models. Our experiments demonstrate the validity of both concepts. First, fine-tuning adapters with program repair specific code representations enables the model to use meaningful repair signals. Second, parameter-efficient fine-tuning helps fine-tuning to converge and contributes to the effectiveness of the repair adapter to fix data-points outside the fine-tuning data distribution. Overall, RepairLLaMA correctly fixes 125 Defects4J v2 and 82 HumanEval-Java bugs, outperforming all baselines.

翻译：自动化程序修复（APR）随着大语言模型（LLM）的出现取得了显著进展。针对程序修复任务对LLM进行微调是近年来的研究方向，但其中许多维度尚未被探索。现有工作大多采用朴素代码表示对LLM进行微调，且从根本上受限于更大型LLM的微调能力。为解决这一问题，我们提出RepairLLaMA——一种新型程序修复方法，该方法结合了1）面向APR的代码表示与2）当前最先进的参数高效LLM微调技术LoRA。这使得RepairLLaMA能够生成高效的“程序修复适配器”，用于利用语言模型修复缺陷。实验验证了上述两个概念的有效性：第一，使用面向程序修复的专用代码表示微调适配器，能使模型利用有意义的修复信号；第二，参数高效微调有助于微调过程的收敛，并增强了修复适配器修复超出微调数据分布范围的数据点的能力。总体而言，RepairLLaMA正确修复了125个Defects4J v2版本和82个HumanEval-Java缺陷，性能超越所有基线方法。