Large language models (LLMs), pre-trained or fine-tuned on large code corpora, have shown effectiveness in generating code completions. However, in LLM-based code completion, LLMs may struggle to use correct and up-to-date Application Programming Interfaces (APIs) due to the rapid and continuous evolution of libraries. While existing studies have highlighted issues with predicting incorrect APIs, the specific problem of deprecated API usage in LLM-based code completion has not been thoroughly investigated. To address this gap, we conducted the first evaluation study on deprecated API usage in LLM-based code completion. This study involved seven advanced LLMs, 145 API mappings from eight popular Python libraries, and 28,125 completion prompts. The study results reveal the \textit{status quo} and \textit{root causes} of deprecated API usage in LLM-based code completion from the perspectives of \textit{model}, \textit{prompt}, and \textit{library}. Based on these findings, we propose two lightweight fixing approaches, \textsc{ReplaceAPI} and \textsc{InsertPrompt}, which can serve as baseline approaches for future research on mitigating deprecated API usage in LLM-based completion. Additionally, we provide implications for future research on integrating library evolution with LLM-driven software development.
翻译:基于大规模代码语料库预训练或微调的大型语言模型(LLMs)在生成代码补全方面已展现出显著效果。然而,在基于LLM的代码补全过程中,由于库的快速持续演进,LLMs可能难以使用正确且最新的应用程序编程接口(APIs)。现有研究虽已指出预测错误API的问题,但针对基于LLM的代码补全中已弃用API使用这一具体问题尚未得到深入探究。为填补这一空白,我们首次开展了关于基于LLM的代码补全中已弃用API使用的评估研究。本研究涵盖七个先进LLM模型、八个流行Python库中的145组API映射关系以及28,125个补全提示。研究结果从模型、提示和库三个维度揭示了基于LLM的代码补全中已弃用API使用的现状与根本成因。基于这些发现,我们提出了两种轻量级修复方法:\textsc{ReplaceAPI} 和 \textsc{InsertPrompt},可作为未来研究缓解基于LLM的补全中已弃用API使用问题的基线方法。此外,我们为未来整合库演进与LLM驱动软件开发的研究提供了启示。