Document understanding tasks, in particular, Visually-rich Document Entity Retrieval (VDER), have gained significant attention in recent years thanks to their broad applications in enterprise AI. However, publicly available data have been scarce for these tasks due to strict privacy constraints and high annotation costs. To make things worse, the non-overlapping entity spaces from different datasets hinder the knowledge transfer between document types. In this paper, we propose a method to collect massive-scale and weakly labeled data from the web to benefit the training of VDER models. The collected dataset, named DocumentNet, does not depend on specific document types or entity sets, making it universally applicable to all VDER tasks. The current DocumentNet consists of 30M documents spanning nearly 400 document types organized in a four-level ontology. Experiments on a set of broadly adopted VDER tasks show significant improvements when DocumentNet is incorporated into the pre-training for both classic and few-shot learning settings. With the recent emergence of large language models (LLMs), DocumentNet provides a large data source to extend their multi-modal capabilities for VDER.

翻译：文档理解任务，尤其是视觉丰富的文档实体检索（Visually-rich Document Entity Retrieval, VDER），由于其在企业级人工智能中的广泛应用，近年来受到了显著关注。然而，由于严格的隐私约束和高昂的标注成本，这些任务的公开可用数据一直十分稀缺。更糟糕的是，不同数据集中不重叠的实体空间阻碍了文档类型间的知识迁移。本文提出了一种从网络收集大规模弱标注数据的方法，以促进VDER模型的训练。所收集的数据集名为DocumentNet，它不依赖于特定的文档类型或实体集，因此可普遍适用于所有VDER任务。当前的DocumentNet包含3000万条文档，涵盖近400种文档类型，并按四层本体进行组织。在一系列广泛采用的VDER任务上的实验表明，无论是经典学习还是少样本学习设置中，将DocumentNet纳入预训练均能带来显著改进。随着近期大语言模型（LLMs）的兴起，DocumentNet为扩展其在VDER任务上的多模态能力提供了庞大的数据源。