Software developers often repeat code changes, known as "code change patterns" (CPATs), within and across projects. Automating these CPATs accelerates development, but current Transformation by Example (TBE) techniques are limited by the input examples' quality and quantity, missing variations with different syntax or flow yet semantically similar. Large Language Models (LLMs), trained on vast code datasets, can overcome these limitations by generating semantically equivalent, unseen CPAT variants, enhancing TBE effectiveness. We identified best practices for using LLMs to generate code variants meeting criteria of correctness, usefulness, and applicability. Implementing these in PyCraft, combining static and dynamic analysis with LLMs, we achieved an F-measure of 96.6% in identifying correct variants, expanding inputs by 58x on average, and automating changes to increase target codes by up to 39x. Patches from PyCraft were submitted to projects like microsoft/DeepSpeed and IBM/inFairness, with an 83% acceptance rate, validating our approach's usefulness.
翻译:软件开发人员经常在项目内和跨项目中重复进行代码变更,这些被称为“代码变更模式”(CPATs)。自动化这些CPATs可以加速开发进程,但当前的示例驱动转换(TBE)技术受限于输入示例的质量和数量,无法处理那些语法或流程不同但语义相似的变体。基于海量代码数据训练的大型语言模型(LLMs)能够通过生成语义等价、未见过的CPAT变体来克服这些限制,从而提升TBE的有效性。我们确定了使用LLMs生成满足正确性、实用性和适用性标准的代码变体的最佳实践。通过在PyCraft中实施这些实践,结合静态与动态分析以及LLMs,我们在识别正确变体方面达到了96.6%的F值,平均将输入扩展了58倍,并通过自动化变更使目标代码量增加了高达39倍。PyCraft生成的补丁已提交至如microsoft/DeepSpeed和IBM/inFairness等项目,接受率达到83%,验证了我们方法的实用性。