This paper defines and explores the design space for information extraction (IE) from layout-rich documents using large language models (LLMs). The three core challenges of layout-aware IE with LLMs are 1) data structuring, 2) model engagement, and 3) output refinement. Our study delves into the sub-problems within these core challenges, such as input representation, chunking, prompting, and selection of LLMs and multimodal models. It examines the outcomes of different design choices through a new layout-aware IE test suite, benchmarking against the state-of-art (SoA) model LayoutLMv3. The results show that the configuration from one-factor-at-a-time (OFAT) trial achieves near-optimal results with 14.1 points F1-score gain from the baseline model, while full factorial exploration yields only a slightly higher 15.1 points gain at around 36x greater token usage. We demonstrate that well-configured general-purpose LLMs can match the performance of specialized models, providing a cost-effective alternative. Our test-suite is freely available at https://github.com/gayecolakoglu/LayIE-LLM.

翻译：本文定义并探索了利用大语言模型从富布局文档中进行信息提取的设计空间。使用大语言模型进行布局感知信息提取面临的三个核心挑战是：1）数据结构化，2）模型交互，以及3）输出优化。我们的研究深入探讨了这些核心挑战中的子问题，例如输入表示、分块处理、提示工程以及大语言模型和多模态模型的选择。研究通过一个新的布局感知信息提取测试套件，对比当前最先进的LayoutLMv3模型，检验了不同设计选择的结果。结果表明，通过单因素轮换试验获得的配置方案取得了接近最优的结果，相比基线模型F1分数提升了14.1分，而全因子探索仅带来略高的15.1分提升，但其令牌使用量却增加了约36倍。我们证明，经过良好配置的通用大语言模型能够达到专用模型的性能，提供了一种具有成本效益的替代方案。我们的测试套件已在https://github.com/gayecolakoglu/LayIE-LLM 开源提供。