Large language models (LLMs) pre-trained on massive corpora have demonstrated impressive few-shot learning ability on many NLP tasks. A common practice is to recast the task into a text-to-text format such that generative LLMs of natural language (NL-LLMs) like GPT-3 can be prompted to solve it. However, it is nontrivial to perform information extraction (IE) tasks with NL-LLMs since the output of the IE task is usually structured and therefore is hard to be converted into plain text. In this paper, we propose to recast the structured output in the form of code instead of natural language and utilize generative LLMs of code (Code-LLMs) such as Codex to perform IE tasks, in particular, named entity recognition and relation extraction. In contrast to NL-LLMs, we show that Code-LLMs can be well-aligned with these IE tasks by designing code-style prompts and formulating these IE tasks as code generation tasks. Experiment results on seven benchmarks show that our method consistently outperforms fine-tuning moderate-size pre-trained models specially designed for IE tasks (e.g., UIE) and prompting NL-LLMs under few-shot settings. We further conduct a series of in-depth analyses to demonstrate the merits of leveraging Code-LLMs for IE tasks.

翻译：基于大规模语料预训练的大型语言模型（LLMs）在许多自然语言处理任务中展现出令人印象深刻的少样本学习能力。一种常见做法是将任务重新构造为文本到文本格式，以便可以提示像GPT-3这样的自然语言生成式LLMs（NL-LLMs）来解决它。然而，使用NL-LLMs执行信息抽取（IE）任务并非易事，因为IE任务的输出通常是结构化的，因此难以转换为纯文本。在本文中，我们提出以代码形式而非自然语言形式重构结构化输出，并利用像Codex这样的代码生成式LLMs（Code-LLMs）来执行IE任务，特别是命名实体识别和关系抽取。与NL-LLMs相比，我们通过设计代码风格提示并将这些IE任务制定为代码生成任务，证明Code-LLMs可以很好地与这些IE任务对齐。在七个基准测试上的实验结果表明，在少样本设置下，我们的方法始终优于专门为IE任务设计的微调中等规模预训练模型（例如UIE）以及提示NL-LLMs。我们进一步进行了一系列深入分析，以展示利用Code-LLMs执行IE任务的优势。