Although large language models (LLMs) have demonstrated impressive ability in code generation, they are still struggling to address the complicated intent provided by humans. It is widely acknowledged that humans typically employ planning to decompose complex problems and schedule solution steps prior to implementation. To this end, we introduce planning into code generation to help the model understand complex intent and reduce the difficulty of problem-solving. This paper proposes a self-planning code generation approach with large language models, which consists of two phases, namely planning phase and implementation phase. Specifically, in the planning phase, LLM plans out concise solution steps from the intent combined with few-shot prompting. Subsequently, in the implementation phase, the model generates code step by step, guided by the preceding solution steps. We conduct extensive experiments on various code-generation benchmarks across multiple programming languages. Experimental results show that self-planning code generation achieves a relative improvement of up to 25.4% in Pass@1 compared to direct code generation, and up to 11.9% compared to Chain-of-Thought of code generation. Moreover, our self-planning approach also enhances the quality of the generated code with respect to correctness, readability, and robustness, as assessed by humans.
翻译:尽管大语言模型在代码生成方面展现了令人印象深刻的能力,但在处理人类提供的复杂意图时仍面临挑战。众所周知,人类通常通过规划来分解复杂问题,并在实施前安排解决步骤。为此,我们将规划引入代码生成过程,以帮助模型理解复杂意图并降低问题求解难度。本文提出了一种基于大语言模型的自我规划代码生成方法,该方法包含规划阶段与实施阶段两个环节。具体而言,在规划阶段,大语言模型结合少量示例提示,从意图中规划出简洁的解决步骤;随后,在实施阶段,模型在前述解决步骤的引导下逐步生成代码。我们在涵盖多种编程语言的各类代码生成基准测试上进行了广泛实验。实验结果表明:与直接代码生成相比,自我规划代码生成在Pass@1指标上最高可获得25.4%的相对提升;与代码生成的思维链方法相比,最高可获得11.9%的相对提升。此外,根据人工评估,我们的自我规划方法在生成代码的正确性、可读性和鲁棒性方面也表现出更优的质量。