Multimodal Large Language Models (MLLMs) have achieved great success in Speech-to-Text Translation (S2TT) tasks. However, current research is constrained by two key challenges: language coverage and efficiency. Most of the popular S2TT datasets are substantially English-centric, which restricts the scaling-up of MLLMs' many-to-many translation capabilities. Moreover, the inference speed of MLLMs degrades dramatically when the speech is converted into long sequences (e.g., 750 tokens). To address these limitations, we propose a Multilingual Cost-effective Accelerated Speech-to-Text Translator (MCAT) framework, which includes two innovations. First, a language scaling method that leverages curriculum learning and a data balancing strategy is introduced to extend the language coverage supported by MLLMs to 70 languages and achieve mutual translation among these languages. Second, an optimized speech adapter module is designed to reduce the length of the speech sequence to only 30 tokens. Extensive experiments were conducted on MLLMs of different scales (9B and 27B). The experimental results demonstrate that MCAT not only surpasses state-of-the-art end-to-end models on the FLEURS dataset across 70x69 directions but also enhances inference efficiency. The code and models are released at https://github.com/yxduir/m2m-70.

翻译：多模态大语言模型在语音到文本翻译任务中取得了巨大成功。然而，当前研究受限于两个关键挑战：语种覆盖范围和效率。主流语音翻译数据集多偏重英语，限制了多模态大语言模型多对多翻译能力的扩展。此外，当语音被转换为长序列时，其推理速度会显著下降。为解决这些局限，我们提出多语种低成本加速语音翻译框架，该框架包含两项创新：首先，引入基于课程学习与数据平衡策略的语种扩展方法，将多模态大语言模型支持的语种覆盖范围扩展至70种，并实现这些语种间的互译；其次，设计优化的语音适配器模块，将语音序列长度缩减至仅30个词元。在9B和27B不同规模的多模态大语言模型上进行了广泛实验。结果表明，MCAT不仅在FLEURS数据集的70×69个方向上超越最先进端到端模型，还提升了推理效率。代码与模型已发布于https://github.com/yxduir/m2m-70。