Generative models trained on the output of earlier models lose knowledge across generations, rare knowledge first. The remedies that work all bring in information from outside the loop, such as fresh human data, verifiers, or curators, and the theory of model collapse treats that supply as given. We argue that cultural evolution has a theory of the supply, and that it distinguishes two ways copying loses knowledge. In a fixed world, copies are lossy, as in Henrich's account of skill loss in small populations. In a changing world, even perfect copies go stale, which is Rogers' paradox. We write both as one chain with separately measurable parameters and test it on lineages of language models (word trigrams and a character transformer on two books, and OPT-125m on wikitext-2, on arXiv abstracts written after its pretraining, and on a world of 2,000 fictional facts that change at a set rate). In a fixed world, neutral drift on word frequencies predicts, with no free parameter, the rare-word coverage of every trigram lineage to within 0.03. In the fact world, the model's accuracy falls and its outdated answers rise with the rate of change; the chain calibrated on the static world predicts both for mid- and low-frequency facts to within 0.03 on average, with the rate of change set, not fitted. Models that inherit their weights keep more in a static world and carry more outdated facts in a changing one, as Rogers' theory says of individual memory. Finally we close the loop with the people who write the data. When they can ask the model instead of finding out, the model ends up exactly as reliable as finding out is costly, and in both regimes someone must keep paying to learn; we give the share in closed form, extend it to lossy copying, heterogeneous costs, aggregating models, and users who check, and show when checking is a stable way out.
翻译:暂无翻译