As synthetic data becomes higher quality and proliferates on the internet, machine learning models are increasingly trained on a mix of human- and machine-generated data. Despite the successful stories of using synthetic data for representation learning, using synthetic data for generative model training creates "self-consuming loops" which may lead to training instability or even collapse, unless certain conditions are met. Our paper aims to stabilize self-consuming generative model training. Our theoretical results demonstrate that by introducing an idealized correction function, which maps a data point to be more likely under the true data distribution, self-consuming loops can be made exponentially more stable. We then propose self-correction functions, which rely on expert knowledge (e.g. the laws of physics programmed in a simulator), and aim to approximate the idealized corrector automatically and at scale. We empirically validate the effectiveness of self-correcting self-consuming loops on the challenging human motion synthesis task, and observe that it successfully avoids model collapse, even when the ratio of synthetic data to real data is as high as 100%.

翻译：随着合成数据质量提升并在互联网上广泛传播，机器学习模型越来越多地基于人工与机器生成数据的混合进行训练。尽管使用合成数据进行表征学习已有成功案例，但将其用于生成模型训练时会形成"自消耗循环"，若不满足特定条件，可能导致训练不稳定甚至崩溃。本文旨在稳定自消耗生成模型训练过程。理论结果表明，通过引入理想化的修正函数（将数据点映射至更符合真实数据分布的状态），自消耗循环的稳定性可呈指数级提升。我们进而提出自修正函数，利用专家知识（例如编程进模拟器的物理定律），以自动化方式大规模逼近理想修正器。我们通过具有挑战性的人体运动合成任务实证验证了自纠正自消耗循环的有效性，并观察到即使合成数据与真实数据比例高达100%，该方法仍能成功避免模型崩溃。