For many business applications that require the processing, indexing, and retrieval of professional documents such as legal briefs (in PDF format etc.), it is often essential to classify the pages of any given document into their corresponding types beforehand. Most existing studies in the field of document image classification either focus on single-page documents or treat multiple pages in a document independently. Although in recent years a few techniques have been proposed to exploit the context information from neighboring pages to enhance document page classification, they typically cannot be utilized with large pre-trained language models due to the constraint on input length. In this paper, we present a simple but effective approach that overcomes the above limitation. Specifically, we enhance the input with extra tokens carrying sequential information about previous pages - introducing recurrence - which enables the usage of pre-trained Transformer models like BERT for context-aware page classification. Our experiments conducted on two legal datasets in English and Portuguese respectively show that the proposed approach can significantly improve the performance of document page classification compared to the non-recurrent setup as well as the other context-aware baselines.

翻译：对于许多需要处理、索引和检索法律简报等专业文档（如PDF格式）的商业应用而言，预先将给定文档的页面分类为相应类型通常至关重要。现有文档图像分类领域的大多数研究要么专注于单页文档，要么将文档中的多页独立处理。尽管近年来已有少数技术提出利用相邻页面的上下文信息来增强文档页面分类，但由于输入长度的限制，这些技术通常无法与大规模预训练语言模型结合使用。本文提出了一种简单而有效的方法来克服上述限制。具体而言，我们通过引入额外标记承载前序页面的序列信息（即引入递归机制）来增强输入，从而使得预训练Transformer模型（如BERT）能够用于上下文感知的页面分类。我们在分别基于英语和葡萄牙语的两个法律数据集上进行的实验表明，与无递归设置及其他上下文感知基线方法相比，所提方法能够显著提升文档页面分类的性能。