We are currently in an era of fierce competition among various large language models (LLMs) continuously pushing the boundaries of benchmark performance. However, genuinely assessing the capabilities of these LLMs has become a challenging and critical issue due to potential data contamination, and it wastes dozens of time and effort for researchers and engineers to download and try those contaminated models. To save our precious time, we propose a novel and useful method, Clean-Eval, which mitigates the issue of data contamination and evaluates the LLMs in a cleaner manner. Clean-Eval employs an LLM to paraphrase and back-translate the contaminated data into a candidate set, generating expressions with the same meaning but in different surface forms. A semantic detector is then used to filter the generated low-quality samples to narrow down this candidate set. The best candidate is finally selected from this set based on the BLEURT score. According to human assessment, this best candidate is semantically similar to the original contamination data but expressed differently. All candidates can form a new benchmark to evaluate the model. Our experiments illustrate that Clean-Eval substantially restores the actual evaluation results on contaminated LLMs under both few-shot learning and fine-tuning scenarios.

翻译：当前正处于各类大型语言模型激烈竞争的时代，这些模型不断刷新基准测试性能的边界。然而，由于潜在的数据污染问题，真正评估这些大型语言模型的能力已成为一个既具挑战性又至关重要的问题，同时这导致研究人员和工程师浪费大量时间与精力下载和测试那些受污染的模型。为节省宝贵时间，我们提出了一种新颖实用的方法——Clean-Eval，该方法缓解了数据污染问题，并以更清洁的方式评估大型语言模型。Clean-Eval利用大型语言模型对受污染数据进行释义和反向翻译，生成候选集，产生语义相同但表面形式不同的表述。随后使用语义检测器过滤生成的低质量样本，缩小候选集范围。最终基于BLEURT评分从候选集中选出最佳候选。根据人工评估，该最佳候选与原始污染数据语义相似但表述不同。所有候选可构成新基准来评估模型。实验表明，Clean-Eval在小样本学习和微调场景下均能显著恢复对受污染大型语言模型的实际评估结果。