In this paper, we present GeneGPT, a novel method for teaching large language models (LLMs) to use the Web Application Programming Interfaces (APIs) of the National Center for Biotechnology Information (NCBI) and answer genomics questions. Specifically, we prompt Codex (code-davinci-002) to solve the GeneTuring tests with few-shot URL requests of NCBI API calls as demonstrations for in-context learning. During inference, we stop the decoding once a call request is detected and make the API call with the generated URL. We then append the raw execution results returned by NCBI APIs to the generated texts and continue the generation until the answer is found or another API call is detected. Our preliminary results show that GeneGPT achieves state-of-the-art results on three out of four one-shot tasks and four out of five zero-shot tasks in the GeneTuring dataset. Overall, GeneGPT achieves a macro-average score of 0.76, which is much higher than retrieval-augmented LLMs such as the New Bing (0.44), biomedical LLMs such as BioMedLM (0.08) and BioGPT (0.04), as well as other LLMs such as GPT-3 (0.16) and ChatGPT (0.12).

翻译：本文提出GeneGPT，一种新颖的方法，用于教导大型语言模型（LLMs）使用美国国家生物技术信息中心（NCBI）的网络应用程序编程接口（API），并回答基因组学问题。具体而言，我们通过提供少量NCBI API调用的URL请求示例作为上下文学习的演示，提示Codex（code-davinci-002）解决GeneTuring测试。在推理过程中，一旦检测到调用请求，我们便停止解码，并使用生成的URL执行API调用。随后，将NCBI API返回的原始执行结果附加到生成的文本中，继续生成直到找到答案或检测到另一个API调用。初步结果显示，GeneGPT在GeneTuring数据集的四项单次学习任务中三项和五项零次学习任务中四项达到最优结果。总体而言，GeneGPT的宏平均得分为0.76，远高于检索增强型LLM（如New Bing的0.44）、生物医学LLM（如BioMedLM的0.08和BioGPT的0.04），以及其他LLM（如GPT-3的0.16和ChatGPT的0.12）。