Simultaneous interpretation (SI), the translation of one language to another in real time, starts translation before the original speech has finished. Its evaluation needs to consider both latency and quality. This trade-off is challenging especially for distant word order language pairs such as English and Japanese. To handle this word order gap, interpreters maintain the word order of the source language as much as possible to keep up with original language to minimize its latency while maintaining its quality, whereas in translation reordering happens to keep fluency in the target language. This means outputs synchronized with the source language are desirable based on the real SI situation, and it's a key for further progress in computational SI and simultaneous machine translation (SiMT). In this work, we propose an automatic evaluation metric for SI and SiMT focusing on word order synchronization. Our evaluation metric is based on rank correlation coefficients, leveraging cross-lingual pre-trained language models. Our experimental results on NAIST-SIC-Aligned and JNPC showed our metrics' effectiveness to measure word order synchronization between source and target language.

翻译：同声传译（SI）指将一种语言实时翻译为另一种语言，其翻译过程在原语发言结束前即已开始。其评估需同时考虑延迟与质量。这种权衡对于词序差异较大的语言对（如英语与日语）尤为困难。为处理这种词序差异，译员会尽可能保持源语言词序以紧跟原语节奏，从而在保证质量的同时最小化延迟；而在笔译中则会发生词序重组以保持目标语言的流畅性。这意味着基于真实同传场景，与源语言保持同步的输出是理想的，这也是计算同传与同声机器翻译（SiMT）取得进一步进展的关键。本研究提出一种专注于词序同步的同声传译与同声机器翻译自动评估指标。该评估指标基于秩相关系数，并利用跨语言预训练语言模型。在NAIST-SIC-Aligned和JNPC数据集上的实验结果表明，本指标能有效度量源语言与目标语言之间的词序同步程度。