A major challenge in the practical use of Machine Translation (MT) is that users lack guidance to make informed decisions about when to rely on outputs. Progress in quality estimation research provides techniques to automatically assess MT quality, but these techniques have primarily been evaluated in vitro by comparison against human judgments outside of a specific context of use. This paper evaluates quality estimation feedback in vivo with a human study simulating decision-making in high-stakes medical settings. Using Emergency Department discharge instructions, we study how interventions based on quality estimation versus backtranslation assist physicians in deciding whether to show MT outputs to a patient. We find that quality estimation improves appropriate reliance on MT, but backtranslation helps physicians detect more clinically harmful errors that QE alone often misses.

翻译：机器翻译（MT）实际应用中的一大挑战在于用户缺乏指导，无法在何时信赖输出结果方面做出明智决策。质量估计研究的进展提供了自动评估MT质量的技术，但这些技术主要是在体外通过与特定使用场景外的人工判断进行比较来评估的。本文通过一项模拟高风险医疗场景下决策的人机实验，对质量估计反馈进行了体内评估。利用急诊科出院指导，我们研究了基于质量估计与基于回译的干预措施如何帮助医生决定是否向患者展示MT输出。研究发现，质量估计能提高对MT的适当依赖，但回译有助于医生检测单凭质量估计常遗漏的临床有害错误。