Automating the annotation of scanned documents is challenging, requiring a balance between computational efficiency and accuracy. DocParseNet addresses this by combining deep learning and multi-modal learning to process both text and visual data. This model goes beyond traditional OCR and semantic segmentation, capturing the interplay between text and images to preserve contextual nuances in complex document structures. Our evaluations show that DocParseNet significantly outperforms conventional models, achieving mIoU scores of 49.12 on validation and 49.78 on the test set. This reflects a 58% accuracy improvement over state-of-the-art baseline models and an 18% gain compared to the UNext baseline. Remarkably, DocParseNet achieves these results with only 2.8 million parameters, reducing the model size by approximately 25 times and speeding up training by 5 times compared to other models. These metrics, coupled with a computational efficiency of 0.034 TFLOPs (BS=1), highlight DocParseNet's high performance in document annotation. The model's adaptability and scalability make it well-suited for real-world corporate document processing applications. The code is available at https://github.com/ahmad-shirazi/DocParseNet

翻译：扫描文档的自动化标注具有挑战性，需要在计算效率与准确性之间取得平衡。DocParseNet通过结合深度学习与多模态学习来处理文本和视觉数据，从而应对这一挑战。该模型超越了传统的光学字符识别（OCR）与语义分割方法，能够捕捉文本与图像之间的相互作用，以保留复杂文档结构中的上下文细微差别。我们的评估表明，DocParseNet显著优于传统模型，在验证集和测试集上分别取得了49.12和49.78的mIoU分数。这反映出相较于最先进的基线模型，其准确率提升了58%；与UNext基线相比，则提升了18%。值得注意的是，DocParseNet仅用280万个参数就实现了这些结果，与其他模型相比，模型规模缩小了约25倍，训练速度加快了5倍。这些指标，加上0.034 TFLOPs（BS=1）的计算效率，突显了DocParseNet在文档标注任务中的高性能表现。该模型的适应性和可扩展性使其非常适合现实世界中的企业文档处理应用。相关代码可在 https://github.com/ahmad-shirazi/DocParseNet 获取。