As synthetic data becomes higher quality and proliferates on the internet, machine learning models are increasingly trained on a mix of human- and machine-generated data. Despite the successful stories of using synthetic data for representation learning, using synthetic data for generative model training creates "self-consuming loops" which may lead to training instability or even collapse, unless certain conditions are met. Our paper aims to stabilize self-consuming generative model training. Our theoretical results demonstrate that by introducing an idealized correction function, which maps a data point to be more likely under the true data distribution, self-consuming loops can be made exponentially more stable. We then propose self-correction functions, which rely on expert knowledge (e.g. the laws of physics programmed in a simulator), and aim to approximate the idealized corrector automatically and at scale. We empirically validate the effectiveness of self-correcting self-consuming loops on the challenging human motion synthesis task, and observe that it successfully avoids model collapse, even when the ratio of synthetic data to real data is as high as 100%.

翻译：随着合成数据质量不断提高并在互联网上广泛传播，机器学习模型越来越多地在人类生成数据与机器生成数据的混合体上进行训练。尽管利用合成数据进行表示学习已取得诸多成功案例，但将其用于生成模型训练会形成“自消耗循环”，除非满足特定条件，否则可能导致训练不稳定甚至崩溃。本文旨在稳定自消耗生成模型的训练过程。理论分析表明，通过引入理想化的校正函数——该函数将数据点映射至更符合真实数据分布的状态——可使自消耗循环的稳定性呈指数级提升。我们进而提出自校正函数，其依托专家知识（例如通过模拟器编程实现的物理定律），旨在以自动化、可扩展的方式逼近理想校正器。我们在极具挑战性的人体运动合成任务上实证验证了自校正自消耗循环的有效性，实验表明即使合成数据与真实数据的比例高达100%，该方法仍能成功避免模型崩溃。