The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant aspects of data contamination in natural language processing, where data contamination is understood as situations where evaluation data is included in pre-training corpora used to train large scale models, compromising evaluation results. The workshop fostered a shared task to collect evidence on data contamination in current available datasets and models. The goal of the shared task and associated database is to assist the community in understanding the extent of the problem and to assist researchers in avoiding reporting evaluation results on known contaminated resources. The shared task provides a structured, centralized public database for the collection of contamination evidence, open to contributions from the community via GitHub pool requests. This first compilation paper is based on 566 reported entries over 91 contaminated sources from a total of 23 contributors. The details of the individual contamination events are available in the platform. The platform continues to be online, open to contributions from the community.
翻译:首届数据污染研讨会(CONDA 2024)聚焦于自然语言处理中数据污染的所有相关方面,其中数据污染被理解为评估数据被包含于用于训练大规模模型的预训练语料库中,从而损害评估结果的情形。本次研讨会组织了一项共享任务,旨在收集当前可用数据集与模型中存在数据污染的证据。该共享任务及相关数据库的目标是帮助学界理解该问题的严重程度,并协助研究者避免在已知受污染资源上报告评估结果。共享任务构建了一个结构化、集中化的公共数据库用于收集污染证据,通过GitHub合并请求向社区开放贡献渠道。本首份汇编论文基于23位贡献者提交的91个污染源共计566条报告记录。具体污染事件的详细信息可通过平台查询。该平台将持续在线运行,并向社区开放贡献入口。