For many business applications that require the processing, indexing, and retrieval of professional documents such as legal briefs (in PDF format etc.), it is often essential to classify the pages of any given document into their corresponding types beforehand. Most existing studies in the field of document image classification either focus on single-page documents or treat multiple pages in a document independently. Although in recent years a few techniques have been proposed to exploit the context information from neighboring pages to enhance document page classification, they typically cannot be utilized with large pre-trained language models due to the constraint on input length. In this paper, we present a simple but effective approach that overcomes the above limitation. Specifically, we enhance the input with extra tokens carrying sequential information about previous pages - introducing recurrence - which enables the usage of pre-trained Transformer models like BERT for context-aware page classification. Our experiments conducted on two legal datasets in English and Portuguese respectively show that the proposed approach can significantly improve the performance of document page classification compared to the non-recurrent setup as well as the other context-aware baselines.

翻译：对于许多需要处理、索引和检索专业文档（如法律简报，PDF格式等）的业务应用而言，预先将给定文档的页面分类为相应类型通常是至关重要的。现有文档图像分类研究大多聚焦于单页文档，或将文档中的多个页面独立处理。尽管近年来已有少数技术利用相邻页面的上下文信息来增强文档页面分类，但由于输入长度的限制，它们通常无法与大型预训练语言模型结合使用。本文提出了一种简单而有效的方法来克服上述限制。具体而言，我们通过引入携带先前页面序列信息的额外令牌来增强输入——从而引入循环机制——这使得能够利用像BERT这样的预训练Transformer模型进行上下文感知的页面分类。我们在分别涉及英语和葡萄牙语的两个法律数据集上进行的实验表明，与无循环设置以及其他上下文感知基线相比，所提出的方法能够显著提升文档页面分类的性能。