Data contamination in language model evaluation is increasingly prevalent as the popularity of large language models. It allows models to "cheat" via memorisation instead of displaying true capabilities. Therefore, contamination analysis has became an crucial part of reliable model evaluation to validate results. However, existing contamination analysis is usually conducted internally by LLM developers and often lacks transparency and completeness. This paper present an open source data contamination reports for the Llama series models. We analyse six popular multi-choice QA benchmarks and quantify their overlapping with the training set of Llama. Various levels of contamination ranging from 1\% to 8.7\% are found across benchmarks. Our comparison also reveals that Llama models can gain over 5\% higher accuracy on contaminated subsets versus clean subsets. Data and code are available at: https://github.com/liyucheng09/Contamination_Detector.

翻译：随着大语言模型的普及，语言模型评估中的数据污染问题日益突出。数据污染使模型能够通过记忆而非展现真实能力进行"作弊"。因此，污染分析已成为验证结果的可靠模型评估中的关键环节。然而，现有的污染分析通常由LLM开发者内部进行，往往缺乏透明性与完整性。本文针对Llama系列模型提出了一份开源数据污染报告。我们分析了六个流行的多选题问答基准测试，并量化了其与Llama训练集的重叠程度。在各基准测试中，发现污染程度从1%到8.7%不等。我们的对比还揭示，Llama模型在受污染子集上的准确率可比干净子集高出5%以上。数据和代码开源地址为：https://github.com/liyucheng09/Contamination_Detector。