API documentation, technical blogs and programming Q&A sites contain numerous partial code that can be reused in programming tasks, but often these code are uncompilable due to unresolved names and syntax errors. To facilitate partial code reuse, we propose the Partial Code Reuse Chain (PCR-Chain) for resolving fully-qualified names (FQNs) and fixing last-mile syntax errors in partial code based on a giant large language model (LLM) like ChatGPT. Methodologically, PCR-Chain is backed up by the underlying global-level prompt architecture (which combines three design ideas: hierarchical task breakdown, prompt composition, and a mix of prompt-based AI and non-AI units) and the local-level prompt design. Technically, we propose PCR-Chain, which employs in-context learning rather than symbolic, costly training methods. Experimental results demonstrate that in dynamically-typed languages (Python), PCR-Chain outperforms current state-of-the-art (SOTA) 5% accuracy like RING. For statically-type languages (Java), our approach achieves high accuracy of 80.5% in resolving both non-FQNs and last-mile syntax errors, surpassing SOTA methods (RING) that can only address last-mile syntax errors. The correct execution of the unit, module, and PCR-Chain demonstrates the effectiveness of the prompt design, composition, and architecture and opens up possibilities for building software engineering tools based on LLMs, replacing traditional program analysis methods.
翻译:API文档、技术博客及编程问答网站中包含大量可复用于编程任务的片段代码,但这些代码常因未解析的命名和语法错误而无法编译。为促进部分代码复用,我们提出基于ChatGPT等大型语言模型(LLM)的部分代码复用链(PCR-Chain),用于解析完全限定名(FQN)并修复部分代码中的末位语法错误。方法层面,PCR-Chain依托全局级提示架构(融合了层级任务分解、提示组合、基于提示的AI与非AI单元混合三种设计思想)和局部级提示设计。技术层面,我们提出的PCR-Chain采用上下文学习范式,而非符号化、高成本的训练方法。实验结果表明:在动态类型语言(Python)中,PCR-Chain相比当前最先进(SOTA)的RING方法准确率提升5%;对于静态类型语言(Java),本方法在同时解析非完全限定名(non-FQN)和修复末位语法错误时达到80.5%的高准确率,超越了仅能处理末位语法错误的SOTA方法(RING)。各单元、模块及PCR-Chain的成功执行验证了提示设计、组合与架构的有效性,为基于LLM构建替代传统程序分析方法的软件工程工具开辟了新可能。