As diffusion probabilistic models (DPMs) are being employed as mainstream models for Generative Artificial Intelligence (GenAI), the study of their memorization of training data has attracted growing attention. Existing works in this direction aim to establish an understanding of whether or to what extent DPMs learn via memorization. Such an understanding is crucial for identifying potential risks of data leakage and copyright infringement in diffusion models and, more importantly, for trustworthy application of GenAI. Existing works revealed that conditional DPMs are more prone to training data memorization than unconditional DPMs, and the motivated data extraction methods are mostly for conditional DPMs. However, these understandings are primarily empirical, and extracting training data from unconditional models has been found to be extremely challenging. In this work, we provide a theoretical understanding of memorization in both conditional and unconditional DPMs under the assumption of model convergence. Our theoretical analysis indicates that extracting data from unconditional models can also be effective by constructing a proper surrogate condition. Based on this result, we propose a novel data extraction method named \textbf{Surrogate condItional Data Extraction (SIDE)} that leverages a time-dependent classifier trained on the generated data as a surrogate condition to extract training data from unconditional DPMs. Empirical results demonstrate that our SIDE can extract training data in challenging scenarios where previous methods fail, and it is, on average, over 50\% more effective across different scales of the CelebA dataset.

翻译：随着扩散概率模型成为生成式人工智能的主流模型，对其训练数据记忆行为的研究日益受到关注。该领域的现有工作旨在理解扩散模型是否通过记忆进行学习及其程度。这种理解对于识别扩散模型中潜在的数据泄露与版权侵权风险至关重要，更重要的是，这关系到生成式人工智能的可信应用。已有研究表明，条件扩散模型比无条件扩散模型更容易出现训练数据记忆现象，且当前主流的数据提取方法主要针对条件扩散模型。然而，这些认识主要基于经验观察，且从无条件模型中提取训练数据已被证明极具挑战性。本研究在模型收敛的假设下，为条件与无条件扩散模型的记忆机制提供了理论解释。理论分析表明，通过构建合适的代理条件，从无条件模型中提取数据同样可行。基于此发现，我们提出了一种名为\textbf{代理条件数据提取}的新方法，该方法利用在生成数据上训练的时变分类器作为代理条件，从无条件扩散模型中提取训练数据。实验结果表明，在以往方法失效的挑战性场景中，我们的方法仍能成功提取训练数据，且在CelebA数据集的不同规模上平均提取效率提升超过50%。