Recent works have shown that generative models leave traces of their underlying generative process on the generated samples, broadly referred to as fingerprints of a generative model, and have studied their utility in detecting synthetic images from real ones. However, the extend to which these fingerprints can distinguish between various types of synthetic image and help identify the underlying generative process remain under-explored. In particular, the very definition of a fingerprint remains unclear, to our knowledge. To that end, in this work, we formalize the definition of artifact and fingerprint in generative models, propose an algorithm for computing them in practice, and finally study its effectiveness in distinguishing a large array of different generative models. We find that using our proposed definition can significantly improve the performance on the task of identifying the underlying generative process from samples (model attribution) compared to existing methods. Additionally, we study the structure of the fingerprints, and observe that it is very predictive of the effect of different design choices on the generative process.
翻译:近年来的研究表明,生成模型会在生成的样本中留下其生成过程的痕迹,这通常被称为生成模型的指纹,并被用于检测合成图像与真实图像。然而,这些指纹在区分不同类型合成图像以及帮助识别底层生成过程方面的潜力尚未得到充分探索。特别是,据我们所知,指纹本身的定义仍不明确。为此,本工作正式定义了生成模型中伪影和指纹的概念,提出了一种在实践中计算它们的算法,并最终研究了该算法在区分大量不同生成模型方面的有效性。我们发现,与现有方法相比,使用我们提出的定义可以显著提升从样本中识别底层生成过程(模型归因)任务的性能。此外,我们研究了指纹的结构,并观察到指纹对生成过程中不同设计选择的影响具有很强的预测能力。