N-gram matching-based evaluation metrics, such as BLEU and chrF, are widely utilized across a range of natural language generation (NLG) tasks. However, recent studies have revealed a weak correlation between these matching-based metrics and human evaluations, especially when compared with neural-based metrics like BLEURT. In this paper, we conjecture that the performance bottleneck in matching-based metrics may be caused by the limited diversity of references. To address this issue, we propose to utilize \textit{multiple references} to enhance the consistency between these metrics and human evaluations. Within the WMT Metrics benchmarks, we observe that the multi-references F200spBLEU surpasses the conventional single-reference one by an accuracy improvement of 7.2\%. Remarkably, it also exceeds the neural-based BERTscore by an accuracy enhancement of 3.9\%. Moreover, we observe that the data leakage issue in large language models (LLMs) can be mitigated to a large extent by our multi-reference metric. We release the code and data at \url{https://github.com/SefaZeng/LLM-Ref}

翻译：基于N-gram匹配的评估指标（如BLEU和chrF）被广泛应用于各类自然语言生成（NLG）任务中。然而，近期研究表明，这些基于匹配的指标与人工评估之间的相关性较弱，尤其在与基于神经网络的指标（如BLEURT）对比时更为明显。本文提出，匹配型指标的性能瓶颈可能源于参考译文的多样性不足。为解决该问题，我们建议采用多参考译文来增强这些指标与人工评估的一致性。在WMT指标基准测试中，我们观察到使用多参考的F200spBLEU相比传统单参考方案获得了7.2%的准确率提升。值得注意的是，该指标还以3.9%的准确率优势超越了基于神经网络的BERTscore。此外，我们发现大型语言模型（LLM）中的数据泄露问题可通过我们的多参考指标得到显著缓解。代码与数据已发布于：\url{https://github.com/SefaZeng/LLM-Ref}