We propose end-to-end document classification and key information extraction (KIE) for automating document processing in forms. Through accurate document classification we harness known information from templates to enhance KIE from forms. We use text and layout encoding with a cosine similarity measure to classify visually-similar documents. We then demonstrate a novel application of mixed integer programming by using assignment optimization to extract key information from documents. Our approach is validated on an in-house dataset of noisy scanned forms. The best performing document classification approach achieved 0.97 f1 score. A mean f1 score of 0.94 for the KIE task suggests there is significant potential in applying optimization techniques. Abation results show that the method relies on document preprocessing techniques to mitigate Type II errors and achieve optimal performance.

翻译：我们提出了一种端到端的文档分类与关键信息抽取（KIE）方法，旨在自动化表单文档处理流程。通过准确的文档分类，我们利用模板中的已知信息来增强表单关键信息抽取的效果。我们采用文本与布局编码结合余弦相似度度量对视觉相似的文档进行分类，并首次展示了混合整数规划的一种新型应用：通过分配优化从文档中提取关键信息。该方法在一个包含噪声扫描表单的内部数据集上进行了验证。最佳文档分类方法达到了0.97的F1分数。关键信息抽取任务的平均F1分数为0.94，表明优化技术具有显著的应用潜力。消融实验结果显示，该方法依赖文档预处理技术来减少第二类错误并实现最优性能。