Recently, deep learning techniques have shown great success in automatic code generation. Inspired by the code reuse, some researchers propose copy-based approaches that can copy the content from similar code snippets to obtain better performance. Practically, human developers recognize the content in the similar code that is relevant to their needs, which can be viewed as a code sketch. The sketch is further edited to the desired code. However, existing copy-based approaches ignore the code sketches and tend to repeat the similar code without necessary modifications, which leads to generating wrong results. In this paper, we propose a sketch-based code generation approach named SkCoder to mimic developers' code reuse behavior. Given a natural language requirement, SkCoder retrieves a similar code snippet, extracts relevant parts as a code sketch, and edits the sketch into the desired code. Our motivations are that the extracted sketch provides a well-formed pattern for telling models "how to write". The post-editing further adds requirement-specific details to the sketch and outputs the complete code. We conduct experiments on two public datasets and a new dataset collected by this work. We compare our approach to 20 baselines using 5 widely used metrics. Experimental results show that (1) SkCoder can generate more correct programs, and outperforms the state-of-the-art - CodeT5-base by 30.30%, 35.39%, and 29.62% on three datasets. (2) Our approach is effective to multiple code generation models and improves them by up to 120.1% in Pass@1. (3) We investigate three plausible code sketches and discuss the importance of sketches. (4) We manually evaluate the generated code and prove the superiority of our SkCoder in three aspects.
翻译:近年来,深度学习技术在自动代码生成领域取得了巨大成功。受代码复用思想的启发,部分研究者提出了基于复制的方案,通过从相似代码片段中复制内容来提升性能。在实际开发中,人类程序员能够识别相似代码中与需求相关的内容,这些内容可视为一种代码草图(code sketch),随后将其编辑为目标代码。然而,现有基于复制的方法忽视了代码草图,倾向于直接重复相似代码而缺乏必要修改,导致生成错误结果。本文提出一种名为SkCoder的基于草图的代码生成方法,模拟程序员的代码复用行为。给定自然语言需求后,SkCoder首先检索相似代码片段,提取相关部分作为代码草图,再将草图编辑为目标代码。我们的动机在于:提取的草图提供了规范的模式,能够指导模型"如何编写";后续编辑阶段则为草图补充需求特定的细节,最终输出完整代码。我们在两个公开数据集及本文新构建的数据集上开展实验,使用5种广泛应用的指标与20个基线方法进行对比。实验结果表明:(1)SkCoder能生成更正确的程序,在三个数据集上分别比当前最优方法CodeT5-base提升30.30%、35.39%和29.62%;(2)该方法对多种代码生成模型均有效,最高可将Pass@1指标提升120.1%;(3)我们研究了三种可行的代码草图方案,并探讨了草图的重要性;(4)通过人工评估生成代码,从三个维度验证了SkCoder的优越性。