Large language models (LLMs) with billions of parameters and pretrained on massive amounts of data are now capable of near or better than state-of-the-art performance in a variety of downstream natural language processing tasks. Neural machine translation (NMT) is one such task that LLMs have been applied to with great success. However, little research has focused on applying LLMs to the more difficult subset of NMT called simultaneous translation (SimulMT), where translation begins before the entire source context is available to the model. In this paper, we address key challenges facing LLMs fine-tuned for SimulMT, validate classical SimulMT concepts and practices in the context of LLMs, explore adapting LLMs that are fine-tuned for NMT to the task of SimulMT, and introduce Simul-LLM, the first open-source fine-tuning and evaluation pipeline development framework for LLMs focused on SimulMT.
翻译:大语言模型(LLMs)拥有数十亿参数,并在海量数据上预训练,现已在多种下游自然语言处理任务中展现出接近或超越现有最优水平的性能。神经机器翻译(NMT)便是LLMs成功应用的典型任务之一。然而,目前鲜有研究将LLMs应用于更为复杂的子领域——同声传译(SimulMT),即模型在获取完整源语言上下文之前就开始翻译。本文针对为同声传译微调的LLMs所面临的关键挑战展开研究,验证了经典同声传译概念与做法在LLMs语境下的适用性,探索了将已针对NMT微调的LLMs适配至同声传译任务的方法,并提出了Simul-LLM——首个面向同声传译的LLMs开源微调与评估流程开发框架。