Large language models (LLMs) are increasingly used in the creation of online content, creating feedback loops as subsequent generations of models will be trained on this synthetic data. Such loops were shown to lead to distribution shifts - models misrepresenting the true underlying distributions of human data (also called model collapse). However, how human data properties affect such shifts remains poorly understood. In this paper, we provide the first empirical examination of the effect of such properties on the outcome of recursive training. We first confirm that using different human datasets leads to distribution shifts of different magnitudes. Through exhaustive manipulation of dataset properties combined with regression analyses, we then identify a set of properties predicting distribution shift magnitudes. Lexical diversity is found to amplify these shifts, while semantic diversity and data quality mitigate them. Furthermore, we find that these influences are highly modular: data scrapped from a given internet domain has little influence on the content generated for another domain. Finally, experiments on political bias reveal that human data properties affect whether the initial bias will be amplified or reduced. Overall, our results portray a novel view, where different parts of internet may undergo different types of distribution shift.

翻译：大语言模型（LLMs）日益广泛地应用于在线内容生成，由此形成反馈循环，因为后续模型迭代将基于这些合成数据进行训练。此类循环已被证明会导致分布偏移——即模型错误表征人类数据的真实底层分布（亦称为模型崩溃）。然而，人类数据特性如何影响此类偏移仍不甚明晰。本文首次通过实证研究探讨了此类特性对递归训练结果的影响。我们首先证实，使用不同的人类数据集会导致不同程度的分布偏移。通过对数据集特性进行系统性操控并结合回归分析，我们进而识别出一组能够预测分布偏移程度的特性。研究发现词汇多样性会加剧这些偏移，而语义多样性与数据质量则能缓解偏移。此外，这些影响呈现出高度模块化特征：从特定互联网领域抓取的数据对另一领域生成内容的影响甚微。最后，针对政治偏见的实验表明，人类数据特性会影响初始偏见是被放大还是削弱。总体而言，我们的研究结果揭示了一种新图景：互联网的不同领域可能经历不同类型的分布偏移。