Foundation models are trained on increasingly immense and opaque datasets. Even while these models are now key in AI system building, it can be difficult to answer the straightforward question: has the model already encountered a given example during training? We therefore propose a widespread adoption of Data Portraits: artifacts that record training data and allow for downstream inspection. First we outline the properties of such an artifact and discuss how existing solutions can be used to increase transparency. We then propose and implement a solution based on data sketching, stressing fast and space efficient querying. Using our tools, we document a popular language modeling corpus (The Pile) and a recently released code modeling dataset (The Stack). We show that our solution enables answering questions about test set leakage and model plagiarism. Our tool is lightweight and fast, costing only 3% of the dataset size in overhead. We release a live interface of our tools at https://dataportraits.org/ and call on dataset and model creators to release Data Portraits as a complement to current documentation practices.

翻译：基础模型在日益庞大且不透明的数据集上进行训练。尽管这些模型如今已成为人工智能系统构建的关键，但回答一个直截了当的问题却变得困难：模型在训练过程中是否已经遇到过某个特定示例？因此，我们提议广泛采用数据肖像：一种记录训练数据并允许下游检查的人工制品。首先，我们概述了此类人工制品的属性，并讨论了如何利用现有解决方案来提高透明度。随后，我们提出并实现了一种基于数据草图技术的解决方案，强调快速且空间高效的查询能力。利用我们的工具，我们记录了流行的语言建模语料库（The Pile）和近期发布的代码建模数据集（The Stack）。我们证明，该解决方案能够回答关于测试集泄露和模型抄袭的问题。我们的工具轻量且快速，仅占用数据集大小的3%作为开销。我们在 https://dataportraits.org/ 发布了工具的实时界面，并呼吁数据集和模型创建者发布数据肖像，以补充当前的文档实践。