The predictive uncertainty of machine translation (MT) models is typically used as a quality estimation proxy. In this work, we posit that apart from confidently translating when a single correct translation exists, models should also maintain uncertainty when the input is ambiguous. We use uncertainty to measure gender bias in MT systems. When the source sentence includes a lexeme whose gender is not overtly marked, but whose target-language equivalent requires gender specification, the model must infer the appropriate gender from the context and can be susceptible to biases. Prior work measured bias via gender accuracy, however it cannot be applied to ambiguous cases. Using semantic uncertainty, we are able to assess bias when translating both ambiguous and unambiguous source sentences, and find that high translation accuracy does not correlate with exhibiting uncertainty appropriately, and that debiasing affects the two cases differently.

翻译：机器翻译（MT）模型的预测不确定性通常被用作质量评估的代理指标。在本研究中，我们提出：除了在存在单一正确翻译时进行自信翻译外，模型在输入存在歧义时也应保持不确定性。我们利用不确定性来度量机器翻译系统中的性别偏差。当源语句包含一个性别未被明确标记的词汇，而其目标语对应词需要指定性别时，模型必须从上下文中推断适当性别，并可能受到偏差影响。先前研究通过性别准确率来度量偏差，但该方法无法应用于歧义情况。通过语义不确定性，我们能够评估翻译歧义性和非歧义性源语句时的偏差，并发现高翻译准确率与恰当表现不确定性之间并不相关，且去偏差处理对这两种情况的影响存在差异。