Relation extraction (RE) consistently involves a certain degree of labeled or unlabeled data even if under zero-shot setting. Recent studies have shown that large language models (LLMs) transfer well to new tasks out-of-the-box simply given a natural language prompt, which provides the possibility of extracting relations from text without any data and parameter tuning. This work focuses on the study of exploring LLMs, such as ChatGPT, as zero-shot relation extractors. On the one hand, we analyze the drawbacks of existing RE prompts and attempt to incorporate recent prompt techniques such as chain-of-thought (CoT) to improve zero-shot RE. We propose the summarize-and-ask (\textsc{SumAsk}) prompting, a simple prompt recursively using LLMs to transform RE inputs to the effective question answering (QA) format. On the other hand, we conduct comprehensive experiments on various benchmarks and settings to investigate the capabilities of LLMs on zero-shot RE. Specifically, we have the following findings: (i) \textsc{SumAsk} consistently and significantly improves LLMs performance on different model sizes, benchmarks and settings; (ii) Zero-shot prompting with ChatGPT achieves competitive or superior results compared with zero-shot and fully supervised methods; (iii) LLMs deliver promising performance in extracting overlapping relations; (iv) The performance varies greatly regarding different relations. Different from small language models, LLMs are effective in handling challenge none-of-the-above (NoTA) relation.

翻译：关系抽取（RE）即使是在零样本设定下也通常需要一定程度的标记或未标记数据。近期研究表明，大型语言模型（LLMs）仅通过自然语言提示即可直接迁移至新任务，这为无需任何数据与参数调优即可从文本中抽取关系提供了可能性。本研究聚焦于探索将LLMs（如ChatGPT）作为零样本关系抽取器。一方面，我们分析了现有RE提示的缺陷，并尝试整合思维链（CoT）等近期提示技术改进零样本RE。我们提出"总结并提问"（\textsc{SumAsk}）提示方法，这是一种通过递归使用LLMs将RE输入转化为有效问答（QA）格式的简单提示。另一方面，我们基于多种基准和设定开展了全面实验，以探究LLMs在零样本RE中的能力。具体而言，我们获得以下发现：（i）\textsc{SumAsk}在不同模型规模、基准和设定下均能持续且显著提升LLMs性能；（ii）使用ChatGPT的零样本提示在零样本和全监督方法中达到具有竞争力甚至更优的结果；（iii）LLMs在抽取重叠关系时展现出令人期待的性能；（iv）不同关系对应的性能差异显著。与小语言模型不同，LLMs在处理具有挑战性的"无匹配项"（NoTA）关系时表现更有效。