Recently, deep learning techniques have shown great success in automatic code generation. Inspired by the code reuse, some researchers propose copy-based approaches that can copy the content from similar code snippets to obtain better performance. Practically, human developers recognize the content in the similar code that is relevant to their needs, which can be viewed as a code sketch. The sketch is further edited to the desired code. However, existing copy-based approaches ignore the code sketches and tend to repeat the similar code without necessary modifications, which leads to generating wrong results. In this paper, we propose a sketch-based code generation approach named SkCoder to mimic developers' code reuse behavior. Given a natural language requirement, SkCoder retrieves a similar code snippet, extracts relevant parts as a code sketch, and edits the sketch into the desired code. Our motivations are that the extracted sketch provides a well-formed pattern for telling models "how to write". The post-editing further adds requirement-specific details to the sketch and outputs the complete code. We conduct experiments on two public datasets and a new dataset collected by this work. We compare our approach to 20 baselines using 5 widely used metrics. Experimental results show that (1) SkCoder can generate more correct programs, and outperforms the state-of-the-art - CodeT5-base by 30.30%, 35.39%, and 29.62% on three datasets. (2) Our approach is effective to multiple code generation models and improves them by up to 120.1% in Pass@1. (3) We investigate three plausible code sketches and discuss the importance of sketches. (4) We manually evaluate the generated code and prove the superiority of our SkCoder in three aspects.
翻译:近期,深度学习技术在自动代码生成领域取得了显著成功。受代码复用启发,部分研究者提出基于复制的方案,通过从相似代码片段中复制内容以提升性能。实践中,人类开发者会识别相似代码中与需求相关的部分,这一过程可视为代码草图构建,随后对草图进行编辑以生成目标代码。然而,现有基于复制的方法忽略了代码草图这一关键环节,往往直接重复相似代码而缺乏必要修改,导致生成结果错误。本文提出一种名为SkCoder的草图驱动代码生成方法,模拟开发者的代码复用行为。给定自然语言需求,SkCoder首先检索相似代码片段,提取相关部分作为代码草图,再通过编辑将草图转化为目标代码。其核心动机在于:提取的草图能为模型提供规范的代码撰写范式(“如何编写”),后续编辑阶段则为草图补充需求特定的细节,最终输出完整代码。我们在两个公开数据集及本文收集的新数据集上展开实验,采用5种广泛使用的指标与20个基线方法对比。实验结果表明:(1)SkCoder能生成更正确的程序,在三个数据集上相较于当前最优模型CodeT5-base分别提升30.30%、35.39%和29.62%;(2)该方法对多种代码生成模型均有提升效果,Pass@1指标最高提升达120.1%;(3)我们探究了三种可行的代码草图,并讨论了草图的重要性;(4)通过人工评估生成代码,从三个维度验证了SkCoder的优越性。