We present a large-scale video subtitle translation dataset, BigVideo, to facilitate the study of multi-modality machine translation. Compared with the widely used How2 and VaTeX datasets, BigVideo is more than 10 times larger, consisting of 4.5 million sentence pairs and 9,981 hours of videos. We also introduce two deliberately designed test sets to verify the necessity of visual information: Ambiguous with the presence of ambiguous words, and Unambiguous in which the text context is self-contained for translation. To better model the common semantics shared across texts and videos, we introduce a contrastive learning method in the cross-modal encoder. Extensive experiments on the BigVideo show that: a) Visual information consistently improves the NMT model in terms of BLEU, BLEURT, and COMET on both Ambiguous and Unambiguous test sets. b) Visual information helps disambiguation, compared to the strong text baseline on terminology-targeted scores and human evaluation. Dataset and our implementations are available at https://github.com/DeepLearnXMU/BigVideo-VMT.

翻译：我们提出了一个大规模视频字幕翻译数据集BigVideo，旨在推动多模态机器翻译研究。与广泛使用的How2和VaTeX数据集相比，BigVideo规模大10倍以上，包含450万句对和9,981小时视频内容。我们还设计了两个专用测试集，以验证视觉信息的必要性：包含歧义词的Ambiguous测试集，以及文本语境自足无需视觉辅助的Unambiguous测试集。为更好建模文本与视频间的共享语义，我们在跨模态编码器中引入了对比学习方法。在BigVideo上的大量实验表明：(a) 在Ambiguous和Unambiguous测试集上，视觉信息均能持续提升神经机器翻译模型在BLEU、BLEURT和COMET指标上的表现；(b) 相较于强劲的纯文本基线模型，视觉信息在术语导向评分和人工评估中均有助于消歧。数据集及相关实现请访问 https://github.com/DeepLearnXMU/BigVideo-VMT。