Recently, deep learning techniques have shown great success in automatic code generation. Inspired by the code reuse, some researchers propose copy-based approaches that can copy the content from similar code snippets to obtain better performance. Practically, human developers recognize the content in the similar code that is relevant to their needs, which can be viewed as a code sketch. The sketch is further edited to the desired code. However, existing copy-based approaches ignore the code sketches and tend to repeat the similar code without necessary modifications, which leads to generating wrong results. In this paper, we propose a sketch-based code generation approach named SkCoder to mimic developers' code reuse behavior. Given a natural language requirement, SkCoder retrieves a similar code snippet, extracts relevant parts as a code sketch, and edits the sketch into the desired code. Our motivations are that the extracted sketch provides a well-formed pattern for telling models "how to write". The post-editing further adds requirement-specific details to the sketch and outputs the complete code. We conduct experiments on two public datasets and a new dataset collected by this work. We compare our approach to 20 baselines using 5 widely used metrics. Experimental results show that (1) SkCoder can generate more correct programs, and outperforms the state-of-the-art - CodeT5-base by 30.30%, 35.39%, and 29.62% on three datasets. (2) Our approach is effective to multiple code generation models and improves them by up to 120.1% in Pass@1. (3) We investigate three plausible code sketches and discuss the importance of sketches. (4) We manually evaluate the generated code and prove the superiority of our SkCoder in three aspects.
翻译:近期,深度学习方法在自动代码生成领域取得了显著成功。受代码复用思想的启发,部分研究者提出了基于复制的技术,通过从相似代码片段中复制内容来提升性能。实践中,人类开发者能识别相似代码中与需求相关的内容,这些内容可被视为代码草图,并进一步编辑为所需代码。然而,现有基于复制的方法忽略了代码草图,倾向于直接重复相似代码而不做必要修改,导致生成错误结果。本文提出一种名为SkCoder的基于草图的代码生成方法,以模拟开发者的代码复用行为。给定自然语言需求时,SkCoder会检索相似代码片段,提取相关部分作为代码草图,并将草图编辑为目标代码。我们的动机在于:提取的草图可提供良好的模式,让模型了解“如何编写”;后续编辑则为草图补充需求特定细节,最终输出完整代码。我们在两个公开数据集及本文收集的新数据集上开展实验,采用5种广泛使用的指标与20个基线方法进行对比。实验结果表明:(1)SkCoder能生成更多正确程序,在三个数据集上分别比当前最先进的CodeT5-base提升30.30%、35.39%和29.62%;(2)该方法对多种代码生成模型均有效,其Pass@1指标提升最高达120.1%;(3)我们探究了三种可行的代码草图,并讨论了草图的重要性;(4)我们通过人工评估生成代码,从三个角度验证了SkCoder的优越性。