This study explores the transfer learning capabilities of the TrOCR architecture to Spanish. TrOCR is a transformer-based Optical Character Recognition (OCR) model renowned for its state-of-the-art performance in English benchmarks. Inspired by Li et al. assertion regarding its adaptability to multilingual text recognition, we investigate two distinct approaches to adapt the model to a new language: integrating an English TrOCR encoder with a language specific decoder and train the model on this specific language, and fine-tuning the English base TrOCR model on a new language data. Due to the scarcity of publicly available datasets, we present a resource-efficient pipeline for creating OCR datasets in any language, along with a comprehensive benchmark of the different image generation methods employed with a focus on Visual Rich Documents (VRDs). Additionally, we offer a comparative analysis of the two approaches for the Spanish language, demonstrating that fine-tuning the English TrOCR on Spanish yields superior recognition than the language specific decoder for a fixed dataset size. We evaluate our model employing character and word error rate metrics on a public available printed dataset, comparing the performance against other open-source and cloud OCR spanish models. As far as we know, these resources represent the best open-source model for OCR in Spanish. The Spanish TrOCR models are publicly available on HuggingFace [20] and the code to generate the dataset is available on Github [25].

翻译：本研究探讨了TrOCR架构在西班牙语中的迁移学习能力。TrOCR是一种基于Transformer的光学字符识别（OCR）模型，以其在英语基准测试中的最先进性能而闻名。受Li等人关于其多语言文本识别适应性的论断启发，我们研究了两种将该模型适应新语言的策略：将英语TrOCR编码器与特定语言解码器集成并在该语言上训练模型，以及在新的语言数据上对英语基础TrOCR模型进行微调。由于公开可用数据集的稀缺性，我们提出了一种资源高效的流程，用于创建任何语言的OCR数据集，并对不同图像生成方法进行了全面基准测试，重点关注视觉丰富文档（VRD）。此外，我们对西班牙语的两种方法进行了比较分析，结果表明在固定数据集规模下，对英语TrOCR进行西班牙语微调比使用特定语言解码器具有更优的识别性能。我们使用字符错误率和单词错误率指标在公开的印刷数据集上评估模型，并与其它开源及云端西班牙语OCR模型进行性能比较。据我们所知，这些资源代表了目前最佳的西班牙语开源OCR模型。西班牙TrOCR模型已在HuggingFace平台公开提供[20]，数据集生成代码可在GitHub获取[25]。