Prior study shows that pre-training techniques can boost the performance of visual document understanding (VDU), which typically requires models to gain abilities to perceive and reason both document texts and layouts (e.g., locations of texts and table-cells). To this end, we propose visually guided generative text-layout pre-training, named ViTLP. Given a document image, the model optimizes hierarchical language and layout modeling objectives to generate the interleaved text and layout sequence. In addition, to address the limitation of processing long documents by Transformers, we introduce a straightforward yet effective multi-segment generative pre-training scheme, facilitating ViTLP to process word-intensive documents of any length. ViTLP can function as a native OCR model to localize and recognize texts of document images. Besides, ViTLP can be effectively applied to various downstream VDU tasks. Extensive experiments show that ViTLP achieves competitive performance over existing baselines on benchmark VDU tasks, including information extraction, document classification, and document question answering.

翻译：先前研究表明，预训练技术能够提升视觉文档理解（VDU）的性能，该类技术通常要求模型具备感知和推理文档文本及布局（如文本位置和表格单元格）的能力。为此，我们提出视觉引导的生成式文字-布局预训练方法ViTLP。给定文档图像，模型通过优化分层语言与布局建模目标，生成交错排列的文字与布局序列。此外，针对Transformer处理长文档的局限性，我们引入一种简洁高效的多片段生成式预训练方案，使ViTLP能够处理任意长度的密集型文本文档。ViTLP可作为原生OCR模型实现文档图像的文本定位与识别，同时还能有效应用于多种下游VDU任务。大量实验表明，ViTLP在信息抽取、文档分类及文档问答等标准VDU任务上，相较于现有基线方法取得了具有竞争力的性能表现。