Massively multilingual neural machine translation (MMNMT) has been proven to enhance the translation quality of low-resource languages. In this paper, we empirically investigate the translation robustness of Indonesian-Chinese translation in the face of various naturally occurring noise. To assess this, we create a robustness evaluation benchmark dataset for Indonesian-Chinese translation. This dataset is automatically translated into Chinese using four NLLB-200 models of different sizes. We conduct both automatic and human evaluations. Our in-depth analysis reveal the correlations between translation error types and the types of noise present, how these correlations change across different model sizes, and the relationships between automatic evaluation indicators and human evaluation indicators. The dataset is publicly available at https://github.com/tjunlp-lab/ID-ZH-MTRobustEval.

翻译：大规模多语言神经机器翻译（MMNMT）已被证明能够提升低资源语言的翻译质量。本文通过实证研究，探讨了印尼语-中文翻译在面对各种自然噪声时的鲁棒性。为此，我们构建了一个针对印尼语-中文翻译的鲁棒性评估基准数据集。该数据集使用四种不同规模的NLLB-200模型自动翻译成中文。我们进行了自动评估和人工评估。深入分析揭示了翻译错误类型与噪声类型之间的关联、这些关联在不同模型规模下的变化趋势，以及自动评估指标与人工评估指标之间的关系。该数据集已在 https://github.com/tjunlp-lab/ID-ZH-MTRobustEval 开源。