Although large language models (LLMs) have demonstrated impressive ability in code generation, they are still struggling to address the complicated intent provided by humans. It is widely acknowledged that humans typically employ planning to decompose complex problems and schedule solution steps prior to implementation. To this end, we introduce planning into code generation to help the model understand complex intent and reduce the difficulty of problem-solving. This paper proposes a self-planning code generation approach with large language models, which consists of two phases, namely planning phase and implementation phase. Specifically, in the planning phase, LLM outlines concise and formatted planning steps from the intent. Subsequently, in the implementation phase, the model generates code step by step, guided by the preceding planning steps. We conduct extensive experiments on various code-generation benchmarks across multiple programming languages. Experimental results show that self-planning code generation achieves a relative improvement of up to 25.4% in Pass@1 compared to direct code generation, and up to 11.9% compared to Chain-of-Thought code generation. Moreover, our self-planning approach also enhances the quality of the generated code with respect to correctness, readability, and robustness, as assessed by humans.
翻译:尽管大语言模型(LLMs)在代码生成方面已展现出令人印象深刻的能力,但在处理人类提供的复杂意图时仍面临挑战。众所周知,人类在实现解决方案之前,通常会通过规划来分解复杂问题并安排解决步骤。为此,我们将规划引入代码生成,以帮助模型理解复杂意图并降低问题求解的难度。本文提出了一种基于大语言模型的自我规划代码生成方法,该方法包含两个阶段:规划阶段与实现阶段。具体而言,在规划阶段,LLM根据意图勾勒出简洁且格式化的规划步骤。随后,在实现阶段,模型在前述规划步骤的指导下逐步生成代码。我们在多种编程语言的多个代码生成基准测试上进行了广泛的实验。实验结果表明,与直接代码生成相比,自我规划代码生成在Pass@1指标上实现了高达25.4%的相对提升;与思维链代码生成相比,提升高达11.9%。此外,根据人工评估,我们的自我规划方法在生成代码的正确性、可读性和鲁棒性方面也提升了代码质量。