Recent statements about the impressive capabilities of large language models (LLMs) are usually supported by evaluating on open-access benchmarks. Considering the vast size and wide-ranging sources of LLMs' training data, it could explicitly or implicitly include test data, leading to LLMs being more susceptible to data contamination. However, due to the opacity of training data, the black-box access of models, and the rapid growth of synthetic training data, detecting and mitigating data contamination for LLMs faces significant challenges. In this paper, we propose CDD, which stands for Contamination Detection via output Distribution for LLMs. CDD necessitates only the sampled texts to detect data contamination, by identifying the peakedness of LLM's output distribution. To mitigate the impact of data contamination in evaluation, we also present TED: Trustworthy Evaluation via output Distribution, based on the correction of LLM's output distribution. To facilitate this study, we introduce two benchmarks, i.e., DetCon and ComiEval, for data contamination detection and contamination mitigation evaluation tasks. Extensive experimental results show that CDD achieves the average relative improvements of 21.8\%-30.2\% over other contamination detection approaches in terms of Accuracy, F1 Score, and AUC metrics, and can effectively detect contamination caused by the variants of test data. TED significantly mitigates performance improvements up to 66.9\% attributed to data contamination across 24 settings and 21 contamination degrees. In real-world applications, we reveal that ChatGPT exhibits a high potential to suffer from data contamination on HumanEval benchmark.

翻译：近期关于大型语言模型杰出能力的表述通常基于在开放基准测试上的评估结果。考虑到LLM训练数据规模庞大且来源广泛，测试数据可能显式或隐式地包含其中，导致LLM更易受到数据污染的影响。然而，由于训练数据不透明、模型黑盒访问特性以及合成训练数据的快速增长，检测和缓解LLM的数据污染面临重大挑战。本文提出CDD方法，即通过输出分布检测LLM数据污染。该方法仅需采样文本即可通过识别LLM输出分布的峰度来检测数据污染。为减轻数据污染对评估的影响，我们基于LLM输出分布校正提出TED方法：通过输出分布实现可信评估。本研究引入两个基准数据集DetCon和ComiEval，分别用于数据污染检测任务和污染缓解评估任务。大量实验结果表明，在准确率、F1分数和AUC指标上，CDD相较其他污染检测方法平均提升21.8%-30.2%，并能有效检测测试数据变体引发的污染。TED在24种设置和21种污染程度下，可将数据污染导致的性能提升显著降低66.9%。在实际应用中，我们发现ChatGPT在HumanEval基准测试上具有较高的数据污染风险。