Generative adversarial networks (GANs) usually struggle in learning from highly diverse data, whose underlying manifold is complex. In this work, we revisit the mathematical foundations of GANs, and theoretically reveal that the native adversarial loss for GAN training is insufficient to fix the problem of \textit{subsets with positive Lebesgue measure of the generated data manifold lying out of the real data manifold}. Instead, we find that score matching serves as a promising solution to this issue thanks to its capability of persistently pushing the generated data points towards the real data manifold. We thereby propose to improve the optimization of GANs with score matching regularity (SMaRt). Regarding the empirical evidences, we first design a toy example to show that training GANs by the aid of a ground-truth score function can help reproduce the real data distribution more accurately, and then confirm that our approach can consistently boost the synthesis performance of various state-of-the-art GANs on real-world datasets with pre-trained diffusion models acting as the approximate score function. For instance, when training Aurora on the ImageNet $64\times64$ dataset, we manage to improve FID from 8.87 to 7.11, on par with the performance of one-step consistency model. Code is available at \href{https://github.com/thuxmf/SMaRt}{https://github.com/thuxmf/SMaRt}.

翻译：生成对抗网络（GANs）在处理高度多样化、底层流形结构复杂的数据时通常面临困难。本研究重新审视了GANs的数学基础，并从理论上揭示了：原生对抗损失不足以解决“生成数据流形中具有正勒贝格测度的子集位于真实数据流形之外”的问题。相反，我们发现分数匹配因其能够持续将生成数据点推向真实数据流形的特性，为此问题提供了可行的解决方案。基于此，我们提出通过分数匹配正则化（SMaRt）来改进GANs的优化过程。在实证证据方面，我们首先设计了一个玩具示例，表明借助真实分数函数训练GANs能够更精确地复现真实数据分布；随后通过实验证实，当使用预训练扩散模型作为近似分数函数时，我们的方法能够持续提升各种先进GAN模型在真实数据集上的合成性能。例如，在ImageNet $64\times64$数据集上训练Aurora模型时，我们将FID指标从8.87提升至7.11，达到一步一致性模型的性能水平。代码发布于 \href{https://github.com/thuxmf/SMaRt}{https://github.com/thuxmf/SMaRt}。