Recently, deep learning techniques have shown great success in automatic code generation. Inspired by the code reuse, some researchers propose copy-based approaches that can copy the content from similar code snippets to obtain better performance. Practically, human developers recognize the content in the similar code that is relevant to their needs, which can be viewed as a code sketch. The sketch is further edited to the desired code. However, existing copy-based approaches ignore the code sketches and tend to repeat the similar code without necessary modifications, which leads to generating wrong results. In this paper, we propose a sketch-based code generation approach named SkCoder to mimic developers' code reuse behavior. Given a natural language requirement, SkCoder retrieves a similar code snippet, extracts relevant parts as a code sketch, and edits the sketch into the desired code. Our motivations are that the extracted sketch provides a well-formed pattern for telling models "how to write". The post-editing further adds requirement-specific details to the sketch and outputs the complete code. We conduct experiments on two public datasets and a new dataset collected by this work. We compare our approach to 20 baselines using 5 widely used metrics. Experimental results show that (1) SkCoder can generate more correct programs, and outperforms the state-of-the-art - CodeT5-base by 30.30%, 35.39%, and 29.62% on three datasets. (2) Our approach is effective to multiple code generation models and improves them by up to 120.1% in Pass@1. (3) We investigate three plausible code sketches and discuss the importance of sketches. (4) We manually evaluate the generated code and prove the superiority of our SkCoder in three aspects.
翻译:最近,深度学习技术在自动代码生成领域取得了巨大成功。受代码复用启发,一些研究者提出基于复制的方法,可从相似代码片段中复制内容以获得更优性能。实践中,人类开发者会识别相似代码中与需求相关的内容,这可视作代码草图,并进一步编辑为所需代码。然而,现有的基于复制方法忽略了代码草图,倾向于直接重复相似代码而缺乏必要修改,导致生成错误结果。本文提出一种名为SkCoder的基于草图的代码生成方法,以模拟开发者的代码复用行为。给定自然语言需求,SkCoder检索相似代码片段,提取相关部分作为代码草图,并将草图编辑为期望代码。我们的动机在于,提取的草图提供了良好形成的模式,可用于指导模型“如何编写”。后续编辑阶段则向草图添加需求特定细节,输出完整代码。我们在两个公开数据集及本研究收集的新数据集上开展实验,使用5种广泛采用的指标与20个基线方法对比。实验结果表明:(1)SkCoder能生成更多正确程序,在三个数据集上分别以30.30%、35.39%和29.62%的提升超越当前最优方法CodeT5-base;(2)本方法对多种代码生成模型均有效,在Pass@1指标上最高提升120.1%;(3)我们考察了三种可行的代码草图,并讨论了草图的重要性;(4)我们人工评估生成的代码,证明了SkCoder在三个方面的优越性。