Code generation is widely regarded as a key technique for elevating the automation and ultimate quality of software development. Nevertheless, existing code generation approaches usually concentrate on a single stage of the software development process (i.e., the coding stage) and do not take into consideration other stages that are crucial in reducing complexity and ensuring quality assurance. The organization and conduction of multiple stages in software development require collaborative teamwork. To this end, this paper presents a self-collaboration code generation framework employing large language models (LLMs), exemplified by ChatGPT. Specifically, multiple LLMs play distinct roles through role instructions to form teams, addressing code generation tasks collaboratively and interactively without the need for human intervention. To showcase our framework, we assemble an elementary team consisting of three ChatGPT roles (i.e., analyst, coder, and tester) corresponding to the analysis, coding, and testing stages of software development. We conduct comprehensive experiments on various code-generation benchmarks. The experimental results indicate that self-collaboration code generation improves 29.9%-47.1% relative performance compared to naive direct code generation, achieving state-of-the-art performance and even surpassing GPT-4.
翻译:代码生成被广泛视为提升软件开发自动化水平和最终质量的关键技术。然而,现有代码生成方法通常专注于软件开发过程中的单一阶段(即编码阶段),而未考虑其他在降低复杂性和确保质量保证中至关重要的阶段。软件开发中多个阶段的组织与执行需要团队协作。为此,本文提出了一种基于大型语言模型(LLMs)的自协作代码生成框架,以ChatGPT为例。具体而言,通过角色指令使多个LLMs扮演不同角色组建团队,在无需人工干预的情况下协作式、交互式地完成代码生成任务。为展示该框架,我们组建了一个包含三个ChatGPT角色(即分析师、编码员和测试员)的基础团队,分别对应软件开发的三个阶段:分析、编码和测试。我们在多个代码生成基准上进行了全面实验。实验结果表明,与直接代码生成相比,自协作代码生成的相对性能提升了29.9%至47.1%,达到了最先进水平,甚至超越了GPT-4。