Automated Program Repair (APR) has evolved significantly with the advent of Large Language Models (LLMs). Fine-tuning LLMs for program repair is a recent avenue of research, with many dimensions which have not been explored. Existing work mostly fine-tune LLMs with naive code representations and does not scale to frontier models. To address this problem, we propose RepairLLaMA, a novel program repair approach that 1) identifies optimal code representations for APR with fine-tuned models, and 2) pioneers state-of-the-art parameter-efficient fine-tuning technique (PEFT) for program repair. This results in RepairLLaMA producing a highly effective `program repair adapter' for fixing bugs with AI. Our experiments demonstrate the validity of both concepts. First, fine-tuning adapters with program repair specific code representations enables the model to use meaningful repair signals and produce better patches. Second, parameter-efficient fine-tuning helps fine-tuning to converge and clearly contributes to the effectiveness of RepairLLaMA in fixing bugs outside the fine-tuning data distribution. Overall, RepairLLaMA correctly fixes 144 Defects4J v2 and 109 HumanEval-Java bugs, outperforming all baselines.

翻译：自动程序修复（APR）随着大型语言模型（LLM）的出现取得了显著进展。针对程序修复任务微调LLM是近期研究的新方向，其中诸多维度尚未被探索。现有工作大多采用朴素代码表示对LLM进行微调，且无法扩展至前沿模型。为解决此问题，我们提出RepairLLaMA这一新型程序修复方法，该方法：1）为基于微调模型的APR识别最优代码表示，2）率先将先进的参数高效微调技术（PEFT）应用于程序修复。由此，RepairLLaMA可生成高效的"程序修复适配器"，用于人工智能驱动的缺陷修复。实验验证了这两个概念的有效性。首先，采用面向程序修复的特定代码表示微调适配器，使模型能够利用有意义的修复信号生成更优补丁。其次，参数高效微调有助于微调过程收敛，并显著提升RepairLLaMA修复微调数据分布外缺陷的能力。总体而言，RepairLLaMA正确修复了Defects4J v2中的144个缺陷及HumanEval-Java中的109个缺陷，性能优于所有基线方法。