Machine Translation systems can produce different types of errors, some of which are characterized as critical or catastrophic due to the specific negative impact that they can have on users. In this paper we focus on one type of critical error: added toxicity. We evaluate and analyze added toxicity when translating a large evaluation dataset (HOLISTICBIAS, over 472k sentences, covering 13 demographic axes) from English into 164 languages. An automatic toxicity evaluation shows that added toxicity across languages varies from 0% to 5%. The output languages with the most added toxicity tend to be low-resource ones, and the demographic axes with the most added toxicity include sexual orientation, gender and sex, and ability. We also perform human evaluation on a subset of 8 translation directions, confirming the prevalence of true added toxicity. We use a measurement of the amount of source contribution to the translation, where a low source contribution implies hallucination, to interpret what causes toxicity. Making use of the input attributions allows us to explain toxicity, because the source contributions significantly correlate with toxicity for 84% of languages studied. Given our findings, our recommendations to reduce added toxicity are to curate training data to avoid mistranslations, mitigate hallucination and check unstable translations.

翻译：机器翻译系统可能产生不同类型的错误，其中一些由于对用户可能造成的特定负面影响而被定性为严重或灾难性错误。本文聚焦于一类关键错误：新增毒性。我们评估并分析了将大型评估数据集（HOLISTICBIAS，包含超过47.2万条语句，覆盖13个人口统计维度）从英语翻译成164种语言时出现的新增毒性。自动化毒性评估显示，不同语言的新增毒性比例介于0%至5%之间。新增毒性最多的输出语言多为低资源语言，而出现新增毒性最多的人口统计维度包括性取向、性别与性、以及能力。我们还对8个翻译方向进行了人工评估，确认了真实新增毒性的普遍存在。我们采用源语言贡献度测量方法来衡量翻译中源语言信息的贡献量，其中低源语言贡献意味着幻觉，以此解释毒性的成因。利用输入归因方法使我们能够解释毒性现象，因为对于所研究语言的84%，源语言贡献与毒性之间存在显著相关性。基于研究结果，我们提出的减少新增毒性的建议包括：整理训练数据以避免误译、减少幻觉现象并检查不稳定的翻译。