Text image machine translation (TIMT) has been widely used in various real-world applications, which translates source language texts in images into another target language sentence. Existing methods on TIMT are mainly divided into two categories: the recognition-then-translation pipeline model and the end-to-end model. However, how to transfer knowledge from the pipeline model into the end-to-end model remains an unsolved problem. In this paper, we propose a novel Multi-Teacher Knowledge Distillation (MTKD) method to effectively distillate knowledge into the end-to-end TIMT model from the pipeline model. Specifically, three teachers are utilized to improve the performance of the end-to-end TIMT model. The image encoder in the end-to-end TIMT model is optimized with the knowledge distillation guidance from the recognition teacher encoder, while the sequential encoder and decoder are improved by transferring knowledge from the translation sequential and decoder teacher models. Furthermore, both token and sentence-level knowledge distillations are incorporated to better boost the translation performance. Extensive experimental results show that our proposed MTKD effectively improves the text image translation performance and outperforms existing end-to-end and pipeline models with fewer parameters and less decoding time, illustrating that MTKD can take advantage of both pipeline and end-to-end models.

翻译：文本图像机器翻译（TIMT）已被广泛应用于各类实际场景中，其目标是将图像中的源语言文本翻译成另一种目标语言句子。现有TIMT方法主要分为两类：先识别后翻译的流水线模型和端到端模型。然而，如何将流水线模型的知识迁移至端到端模型仍是一个未解决的问题。本文提出一种新颖的多教师知识蒸馏（MTKD）方法，可有效将流水线模型的知识精炼至端到端TIMT模型中。具体而言，我们利用三位教师来提升端到端TIMT模型的性能：通过识别教师编码器的知识蒸馏指导优化端到端TIMT模型中的图像编码器，同时通过迁移翻译序列教师和翻译解码器教师的知识改进序列编码器与解码器。此外，我们融合了词级与句级知识蒸馏以进一步提升翻译性能。大量实验结果表明，所提出的MTKD方法能有效提升文本图像翻译性能，在参数更少、解码时间更短的情况下优于现有端到端与流水线模型，验证了MTKD可兼顾流水线模型与端到端模型的优势。