Automatic API method recommendation is an essential task of code intelligence, which aims to suggest suitable APIs for programming queries. Existing approaches can be categorized into two primary groups: retrieval-based and learning-based approaches. Although these approaches have achieved remarkable success, they still come with notable limitations. The retrieval-based approaches rely on the text representation capabilities of embedding models, while the learning-based approaches require extensive task-specific labeled data for training. To mitigate the limitations, we propose APIGen, a generative API recommendation approach through enhanced in-context learning (ICL). APIGen involves two main components: (1) Diverse Examples Selection. APIGen searches for similar posts to the programming queries from the lexical, syntactical, and semantic perspectives, providing more informative examples for ICL. (2) Guided API Recommendation. APIGen enables large language models (LLMs) to perform reasoning before generating API recommendations, where the reasoning involves fine-grained matching between the task intent behind the queries and the factual knowledge of the APIs. With the reasoning process, APIGen makes recommended APIs better meet the programming requirement of queries and also enhances the interpretability of results. We compare APIGen with four existing approaches on two publicly available benchmarks. Experiments show that APIGen outperforms the best baseline CLEAR by 105.8% in method-level API recommendation and 54.3% in class-level API recommendation in terms of SuccessRate@1. Besides, APIGen achieves an average 49.87% increase compared to the zero-shot performance of popular LLMs such as GPT-4 in method-level API recommendation regarding the SuccessRate@3 metric.
翻译:自动API方法推荐是代码智能的一项重要任务,旨在为编程查询推荐合适的API。现有方法主要分为两类:基于检索的方法和基于学习的方法。尽管这些方法取得了显著成功,但仍存在明显局限性。基于检索的方法依赖于嵌入模型的文本表示能力,而基于学习的方法则需要大量特定任务的标注数据进行训练。为克服这些局限性,我们提出APIGen——一种通过增强上下文学习(ICL)实现的生成式API推荐方法。APIGen包含两个主要组件:(1)多样示例选取。APIGen从词汇、句法和语义角度搜索与编程查询相似的帖子,为ICL提供更具信息量的示例。(2)引导式API推荐。APIGen使大语言模型(LLM)在生成API推荐前进行推理,该推理涉及查询背后的任务意图与API事实知识之间的细粒度匹配。通过推理过程,APIGen使推荐的API更好地满足查询的编程需求,同时增强结果的可解释性。我们在两个公开基准上将APIGen与四种现有方法进行对比。实验表明,在方法级API推荐中,APIGen在SuccessRate@1指标上比最优基线CLEAR提升105.8%;在类级API推荐中提升54.3%。此外,在方法级API推荐的SuccessRate@3指标上,APIGen相比GPT-4等流行LLM的零样本性能平均提升49.87%。