Large language models (LLMs) have achieved decent results on automated program repair (APR). However, the next token prediction training objective of decoder-only LLMs (e.g., GPT-4) is misaligned with the masked span prediction objective of current infilling-style methods, which impedes LLMs from fully leveraging pre-trained knowledge for program repair. In addition, while some LLMs are capable of locating and repairing bugs end-to-end when using the related artifacts (e.g., test cases) as input, existing methods regard them as separate tasks and ask LLMs to generate patches at fixed locations. This restriction hinders LLMs from exploring potential patches beyond the given locations. In this paper, we investigate a new approach to adapt LLMs to program repair. Our core insight is that LLM's APR capability can be greatly improved by simply aligning the output to their training objective and allowing them to refine the whole program without first performing fault localization. Based on this insight, we designed D4C, a straightforward prompting framework for APR. D4C can repair 180 bugs correctly in Defects4J, with each patch being sampled only 10 times. This surpasses the SOTA APR methods with perfect fault localization by 10% and reduces the patch sampling number by 90%. Our findings reveal that (1) objective alignment is crucial for fully exploiting LLM's pre-trained capability, and (2) replacing the traditional localize-then-repair workflow with direct debugging is more effective for LLM-based APR methods. Thus, we believe this paper introduces a new mindset for harnessing LLMs in APR.

翻译：大型语言模型（LLMs）在自动化程序修复（APR）中已取得不错的效果。然而，解码器专用LLMs（如GPT-4）的下一词元预测训练目标与当前填充风格方法的掩码跨度预测目标不匹配，这阻碍了LLMs充分利用预训练知识进行程序修复。此外，尽管某些LLMs能够利用相关工件（如测试用例）作为输入，以端到端的方式定位并修复缺陷，但现有方法将其视为独立任务，且要求LLMs在固定位置生成补丁。这种限制阻碍了LLMs探索给定位置之外的潜在补丁。本文研究了一种使LLMs适配程序修复的新方法。我们的核心洞见是：仅需将其输出与训练目标对齐，并允许其在不预先进行故障定位的情况下重构整个程序，即可显著提升LLMs的APR能力。基于此，我们设计了D4C——一个简洁的APR提示框架。D4C在Defects4J中能正确修复180个缺陷，且每个补丁仅需采样10次。这比具有完美故障定位的当前最优APR方法提升了10%，并将补丁采样数量减少了90%。我们的发现表明：(1) 目标对齐对于充分发挥LLMs预训练能力至关重要；(2) 用直接调试替代传统的"先定位后修复"工作流，对基于LLMs的APR方法更为有效。因此，我们认为本文为在APR中利用LLMs引入了新的思维范式。