Sign language gloss translation aims to translate the sign glosses into spoken language texts, which is challenging due to the scarcity of labeled gloss-text parallel data. Back translation (BT), which generates pseudo-parallel data by translating in-domain spoken language texts into sign glosses, has been applied to alleviate the data scarcity problem. However, the lack of large-scale high-quality domain spoken language text data limits the effect of BT. In this paper, to overcome the limitation, we propose a Prompt based domain text Generation (PGEN) approach to produce the large-scale in-domain spoken language text data. Specifically, PGEN randomly concatenates sentences from the original in-domain spoken language text data as prompts to induce a pre-trained language model (i.e., GPT-2) to generate spoken language texts in a similar style. Experimental results on three benchmarks of sign language gloss translation in varied languages demonstrate that BT with spoken language texts generated by PGEN significantly outperforms the compared methods. In addition, as the scale of spoken language texts generated by PGEN increases, the BT technique can achieve further improvements, demonstrating the effectiveness of our approach. We release the code and data for facilitating future research in this field.
翻译:手语标注翻译旨在将手语标注序列转换为口语文本,由于缺乏标注-口语文本的平行数据,该任务面临挑战。回译(BT)通过将领域内口语文本翻译为手语标注以生成伪平行数据,已被用于缓解数据稀缺问题。然而,缺乏大规模高质量领域口语文本数据限制了回译的效果。为克服这一限制,本文提出基于提示的领域文本生成(PGEN)方法,用于生成大规模领域内口语文本数据。具体而言,PGEN将原始领域内口语文本数据中的句子随机拼接作为提示,以引导预训练语言模型(即GPT-2)生成风格相似的口语文本。在三种不同语言的手语标注翻译基准上的实验结果表明,使用PGEN生成的领域口语文本进行回译,其性能显著优于对比方法。此外,随着PGEN生成的口语文本规模增加,回译技术能够实现进一步改进,这证明了我们方法的有效性。我们已公开代码和数据以促进该领域的未来研究。