Software developers often repeat code changes, known as "code change patterns" (CPATs), within and across projects. Automating these CPATs accelerates development, but current Transformation by Example (TBE) techniques are limited by the input examples' quality and quantity, missing variations with different syntax or flow yet semantically similar. Large Language Models (LLMs), trained on vast code datasets, can overcome these limitations by generating semantically equivalent, unseen CPAT variants, enhancing TBE effectiveness. We identified best practices for using LLMs to generate code variants meeting criteria of correctness, usefulness, and applicability. Implementing these in PyCraft, combining static and dynamic analysis with LLMs, we achieved an F-measure of 96.6% in identifying correct variants, expanding inputs by 58x on average, and automating changes to increase target codes by up to 39x. Patches from PyCraft were submitted to projects like microsoft/DeepSpeed and IBM/inFairness, with an 83% acceptance rate, validating our approach's usefulness.
翻译:软件开发者经常在项目内或跨项目中重复执行代码变更,即所谓的“代码变更模式”(CPATs)。自动化这些CPATs可以加速开发进程,但当前的“示例转换”(TBE)技术受限于输入示例的质量和数量,无法覆盖在语法或流程上存在差异、但语义相似的变体。基于大规模代码数据集训练的大语言模型(LLMs)能够通过生成语义等价且未见过的CPAT变体,来克服这些局限性,从而提升TBE的有效性。我们确定了使用LLMs生成符合正确性、实用性和适用性标准的代码变体的最佳实践。将这些方法集成到PyCraft中,并结合静态分析、动态分析与LLMs,我们在识别正确变体时达到了96.6%的F值,输入平均扩展了58倍,自动化变更使目标代码量增加了多达39倍。来自PyCraft的补丁已被提交至microsoft/DeepSpeed和IBM/inFairness等项目,接受率达到83%,验证了我们方法的实用性。