Although Large Language Models (LLMs) have made significant progress in code generation, they still struggle with code generation tasks in specific scenarios. These scenarios usually necessitate the adaptation of LLMs to fulfill specific needs, but the limited training data available in practice leads to poor code generation performance. Therefore, how to effectively adapt LLMs to new scenarios with few training data is a major challenge for current code generation. In this paper, we propose a novel adaptation approach named DEED, which stands for Data-Efficient adaptation with Error-Driven learning for code generation. DEED leverages the errors made by LLMs as learning opportunities, using error revision to overcome their own shortcomings, thus achieving efficient learning. Specifically, DEED involves identifying error code generated by LLMs, employing Self-Revise for code revision, optimizing the model with revised code, and iteratively adapting the process for continuous improvement. Experimental results show that, compared to other mainstream fine-tuning approaches, DEED achieves superior performance with few training data, showing an average relative improvement of 46.2% in Pass@1 on multiple code generation benchmarks. We also validate the effectiveness of Self-Revise, which generates revised code that optimizes the model more efficiently compared to the code samples from datasets. Moreover, DEED consistently demonstrates strong performance across various LLMs, underscoring its applicability.

翻译：尽管大型语言模型在代码生成方面取得了显著进展，但在特定场景下的代码生成任务中仍面临挑战。这些场景通常需要对大型语言模型进行适应性调整以满足特定需求，但实践中有限的训练数据导致代码生成性能不佳。因此，如何利用少量训练数据有效使大型语言模型适应新场景，是当前代码生成领域面临的主要挑战。本文提出了一种名为DEED的新型适应方法，该方法代表基于错误驱动学习的代码生成数据高效适应。DEED将大型语言模型产生的错误转化为学习机会，通过错误修订来克服自身缺陷，从而实现高效学习。具体而言，DEED包含识别大型语言模型生成的错误代码、采用Self-Revise进行代码修订、使用修订后的代码优化模型，并通过迭代适应过程实现持续改进。实验结果表明，与其他主流微调方法相比，DEED在少量训练数据下实现了更优性能，在多个代码生成基准测试中Pass@1指标平均相对提升46.2%。我们还验证了Self-Revise的有效性，其生成的修订代码相较于数据集中的代码样本能更高效地优化模型。此外，DEED在不同大型语言模型上均表现出稳定的优异性能，证明了该方法的广泛适用性。