We present the GPT-NL Public Corpus, the biggest permissively licensed corpus of Dutch language resources. The GPT-NL Public Corpus contains 21 Dutch-only collections totalling 36B preprocessed Dutch tokens not present in any other LLM pretraining corpus. Additionally, the corpus includes roughly 207B English, 232B Code, and 48B German/Danish tokens taken from existing sets which we further curated for compliance. This corpus includes curated data from large existing corpora like Common Corpus and Common Crawl, as well as newly created Dutch-specific collections. Most newly created Dutch collections consist of content collected in collaboration with organisations or synthetically augmented content. All data is collected and evaluated with the aim of facilitating the creation of (commercial) language models that are lawful, useful and non-harmful. All data included in the GPT-NL Public Corpus is sourced from datasets with permissive licensing and is curated and redistributed under a CC-BY license. The full dataset is publicly available on the Hugging Face Hub.

翻译：我们发布了GPT-NL公共语料库，这是目前规模最大的采用宽松许可的荷兰语资源语料库。该语料库包含21个纯荷兰语数据集，总计360亿预处理后的荷兰语词元，这些词元未出现在任何其他大型语言模型预训练语料库中。此外，语料库还纳入了约2070亿英语词元、2320亿代码词元以及480亿德语/丹麦语词元，这些词元来自我们为进一步确保合规性而精选的现有数据集。本语料库整合了来自Common Corpus和Common Crawl等大型现有语料库的精选数据，以及新创建的荷兰语专用数据集。大多数新建荷兰语数据集包含与机构合作收集的内容或通过合成增强生成的内容。所有数据的收集与评估均旨在促进创建合法、实用且无害的（商业）语言模型。GPT-NL公共语料库中包含的所有数据均来源于采用宽松许可的数据集，并在CC-BY许可下进行整理与重新分发。完整数据集已在Hugging Face Hub上公开发布。