Neural metrics trained on human evaluations of MT tend to correlate well with human judgments, but their behavior is not fully understood. In this paper, we perform a controlled experiment and compare a baseline metric that has not been trained on human evaluations (Prism) to a trained version of the same metric (Prism+FT). Surprisingly, we find that Prism+FT becomes more robust to machine-translated references, which are a notorious problem in MT evaluation. This suggests that the effects of metric training go beyond the intended effect of improving overall correlation with human judgments.

翻译：基于人工评估训练的神经度量指标通常与人类判断高度相关，但其行为机制尚未完全明确。本文通过受控实验，将未经过人工评估训练的基线度量指标（Prism）与同一指标的训练版本（Prism+FT）进行对比。令人意外的是，我们发现Prism+FT对机器翻译参考译文的鲁棒性显著增强——这恰恰是机器翻译评估中公认的难题。该结果表明，度量指标训练产生的效应已超越预期的"提升与人类判断的整体相关性"这一原始目标。