Large language models (LLMs), such as GPT-3 and ChatGPT, have demonstrated remarkable results in various natural language processing (NLP) tasks with in-context learning, which involves inference based on a few demonstration examples. Despite their successes in NLP tasks, no investigation has been conducted to assess the ability of LLMs to perform document information extraction (DIE) using in-context learning. Applying LLMs to DIE poses two challenges: the modality and task gap. To this end, we propose a simple but effective in-context learning framework called ICL-D3IE, which enables LLMs to perform DIE with different types of demonstration examples. Specifically, we extract the most difficult and distinct segments from hard training documents as hard demonstrations for benefiting all test instances. We design demonstrations describing relationships that enable LLMs to understand positional relationships. We introduce formatting demonstrations for easy answer extraction. Additionally, the framework improves diverse demonstrations by updating them iteratively. Our experiments on three widely used benchmark datasets demonstrate that the ICL-D3IE framework enables Davinci-003/ChatGPT to achieve superior performance when compared to previous pre-trained methods fine-tuned with full training in both the in-distribution (ID) setting and in the out-of-distribution (OOD) setting. Code is available at https://github.com/MAEHCM/ICL-D3IE.

翻译：大型语言模型（LLMs），如GPT-3和ChatGPT，已通过上下文学习（in-context learning）在多种自然语言处理任务中展现出显著成果，该机制基于少量演示示例进行推理。尽管LLMs在自然语言处理任务中取得突破，但尚未有研究系统评估其通过上下文学习执行文档信息抽取（DIE）的能力。将LLMs应用于DIE面临两大挑战：模态差异与任务差异。为此，我们提出一种简单但有效的上下文学习框架ICL-D3IE，该框架使LLMs能够利用不同类型的演示示例进行DIE。具体而言，我们从困难训练文档中提取最具挑战性和区分度的片段作为硬演示（hard demonstrations），以提升所有测试实例的泛化能力；设计描述位置关系的演示以使LLMs理解空间关联；引入格式化演示以便于答案抽取。此外，该框架通过迭代更新演示来增强多样性。在三个广泛使用的基准数据集上的实验表明，ICL-D3IE框架使Davinci-003/ChatGPT在分布内（ID）与分布外（OOD）设置下，均优于此前使用全量训练数据进行微调的预训练方法。代码已开源至https://github.com/MAEHCM/ICL-D3IE。