Recent statements about the impressive capabilities of large language models (LLMs) are usually supported by evaluating on open-access benchmarks. Considering the vast size and wide-ranging sources of LLMs' training data, it could explicitly or implicitly include test data, leading to LLMs being more susceptible to data contamination. However, due to the opacity of training data, the black-box access of models, and the rapid growth of synthetic training data, detecting and mitigating data contamination for LLMs faces significant challenges. In this paper, we propose CDD, which stands for Contamination Detection via output Distribution for LLMs. CDD necessitates only the sampled texts to detect data contamination, by identifying the peakedness of LLM's output distribution. To mitigate the impact of data contamination in evaluation, we also present TED: Trustworthy Evaluation via output Distribution, based on the correction of LLM's output distribution. To facilitate this study, we introduce two benchmarks, i.e., DetCon and ComiEval, for data contamination detection and contamination mitigation evaluation tasks. Extensive experimental results show that CDD achieves the average relative improvements of 21.8\%-30.2\% over other contamination detection approaches in terms of Accuracy, F1 Score, and AUC metrics, and can effectively detect contamination caused by the variants of test data. TED significantly mitigates performance improvements up to 66.9\% attributed to data contamination across 24 settings and 21 contamination degrees. In real-world applications, we reveal that ChatGPT exhibits a high potential to suffer from data contamination on HumanEval benchmark.

翻译：近期关于大型语言模型（LLMs）令人印象深刻能力的表述，通常基于在开放获取基准上的评估结果。考虑到LLMs训练数据的庞大规模和广泛来源，其可能显式或隐式地包含测试数据，导致LLMs更容易受到数据污染的影响。然而，由于训练数据的不透明性、模型的黑盒访问方式以及合成训练数据的快速增长，检测和缓解LLMs的数据污染面临重大挑战。在本文中，我们提出CDD，即基于输出分布的数据污染检测方法（Contamination Detection via output Distribution）。CDD仅需采样文本即可通过识别LLM输出分布的峰度来检测数据污染。为减轻评估中数据污染的影响，我们还提出了TED：基于输出分布的可信评估方法（Trustworthy Evaluation via output Distribution），该方法通过对LLM输出分布进行校正实现。为促进本研究，我们引入两个基准数据集——DetCon和ComiEval，分别用于数据污染检测和污染缓解评估任务。大量实验结果表明，在准确率、F1分数和AUC指标上，CDD相较其他污染检测方法实现了21.8%-30.2%的平均相对提升，并能有效检测由测试数据变体引起的污染。在24种设置和21种污染程度下，TED显著缓解了由数据污染导致的性能提升，最高达66.9%。在实际应用中，我们发现ChatGPT在HumanEval基准上极有可能遭受数据污染的影响。