Semantic and Cross-language code clone generation may be useful for code reuse, code comprehension, refactoring and benchmarking. OpenAI's GPT model has potential in such clone generation as GPT is used for text generation. When developers copy/paste codes from Stack Overflow (SO) or within a system, there might be inconsistent changes leading to unexpected behaviours. Similarly, if someone possesses a code snippet in a particular programming language but seeks equivalent functionality in a different language, a semantic cross-language code clone generation approach could provide valuable assistance.In this study, using SemanticCloneBench as a vehicle, we evaluated how well the GPT-3 model could help generate semantic and cross-language clone variants for a given fragment.We have comprised a diverse set of code fragments and assessed GPT-3s performance in generating code variants.Through extensive experimentation and analysis, where 9 judges spent 158 hours to validate, we investigate the model's ability to produce accurate and semantically correct variants. Our findings shed light on GPT-3's strengths in code generation, offering insights into the potential applications and challenges of using advanced language models in software development. Our quantitative analysis yields compelling results. In the realm of semantic clones, GPT-3 attains an impressive accuracy of 62.14% and 0.55 BLEU score, achieved through few-shot prompt engineering. Furthermore, the model shines in transcending linguistic confines, boasting an exceptional 91.25% accuracy in generating cross-language clones
翻译:语义与跨语言代码克隆生成可能有助于代码复用、代码理解、重构及基准测试。OpenAI的GPT模型在此类克隆生成中具有潜力,因其原用于文本生成。当开发者从Stack Overflow或系统内部复制粘贴代码时,可能存在不一致的修改,从而导致意外行为。同样,若某人拥有特定编程语言的代码片段,但需要另一种语言中功能等价的代码,语义跨语言代码克隆生成方法可提供宝贵辅助。本研究以SemanticCloneBench为载体,评估了GPT-3模型在生成给定代码片段的语义与跨语言克隆变体方面的能力。我们构建了一组多样化的代码片段,并评估了GPT-3生成代码变体的表现。通过广泛实验与分析(9名评审员耗时158小时进行验证),我们探究了该模型生成准确且语义正确变体的能力。研究结果揭示了GPT-3在代码生成中的优势,为先进语言模型在软件开发中的潜在应用与挑战提供了见解。定量分析产生了令人信服的结果:在语义克隆领域,GPT-3通过少量提示工程实现了62.14%的准确率与0.55的BLEU分数;此外,该模型在突破语言限制方面表现突出,生成跨语言克隆的准确率高达91.25%。