Recent work has shown that fine-tuning large language models (LLMs) on large-scale instruction-following datasets substantially improves their performance on a wide range of NLP tasks, especially in the zero-shot setting. However, even advanced instruction-tuned LLMs still fail to outperform small LMs on relation extraction (RE), a fundamental information extraction task. We hypothesize that instruction-tuning has been unable to elicit strong RE capabilities in LLMs due to RE's low incidence in instruction-tuning datasets, making up less than 1% of all tasks (Wang et al., 2022). To address this limitation, we propose QA4RE, a framework that aligns RE with question answering (QA), a predominant task in instruction-tuning datasets. Comprehensive zero-shot RE experiments over four datasets with two series of instruction-tuned LLMs (six LLMs in total) demonstrate that our QA4RE framework consistently improves LLM performance, strongly verifying our hypothesis and enabling LLMs to outperform strong zero-shot baselines by a large margin. Additionally, we provide thorough experiments and discussions to show the robustness, few-shot effectiveness, and strong transferability of our QA4RE framework. This work illustrates a promising way of adapting LLMs to challenging and underrepresented tasks by aligning these tasks with more common instruction-tuning tasks like QA.

翻译：近期研究表明，在大规模指令遵循数据集上微调大语言模型（LLMs）可显著提升其在多种自然语言处理任务（尤其在零样本设置下）的性能。然而，即便是先进的指令微调大语言模型，在关系抽取（RE）这一基础信息抽取任务上仍难以超越小型语言模型。我们推测，这是由于指令微调无法充分激发大语言模型的强关系抽取能力——关系抽取任务在指令微调数据集中占比不足全部任务的1%（Wang et al., 2022）。为解决这一局限，我们提出QA4RE框架，该框架将关系抽取与指令微调数据集中的主导任务——问答（QA）对齐。在四个数据集上对两个系列指令微调大语言模型（总计六个大语言模型）进行的零样本关系抽取实验表明，QA4RE框架能持续提升大语言模型性能，有力验证了我们的假设，并使大语言模型以显著优势超越强零样本基线。此外，我们通过充分的实验与讨论展示了QA4RE框架的鲁棒性、少样本有效性及强可迁移性。本研究通过将关系抽取等挑战性低资源任务与问答等更常见的指令微调任务对齐，为将大语言模型适配此类任务提供了有效路径。