Text-to-image diffusion models have achieved tremendous success in the field of controllable image generation, while also coming along with issues of privacy leakage and data copyrights. Membership inference arises in these contexts as a potential auditing method for detecting unauthorized data usage. While some efforts have been made on diffusion models, they are not applicable to text-to-image diffusion models due to the high computation overhead and enhanced generalization capabilities. In this paper, we first identify a conditional overfitting phenomenon in text-to-image diffusion models, indicating that these models tend to overfit the conditional distribution of images given the text rather than the marginal distribution of images. Based on this observation, we derive an analytical indicator, namely Conditional Likelihood Discrepancy (CLiD), to perform membership inference, which reduces the stochasticity in estimating the memorization of individual samples. Experimental results demonstrate that our method significantly outperforms previous methods across various data distributions and scales. Additionally, our method shows superior resistance to overfitting mitigation strategies such as early stopping and data augmentation.

翻译：文本到图像扩散模型在可控图像生成领域取得了巨大成功，同时也伴随着隐私泄露和数据版权问题。在此背景下，成员推断作为一种检测未授权数据使用的潜在审计方法应运而生。尽管已有一些针对扩散模型的研究工作，但由于高计算开销和增强的泛化能力，这些方法并不适用于文本到图像扩散模型。本文首先识别了文本到图像扩散模型中的条件过拟合现象，表明这些模型倾向于过拟合给定文本条件下的图像条件分布，而非图像的边缘分布。基于这一观察，我们推导出一个分析性指标——条件似然差异（CLiD），用于执行成员推断，该指标降低了估计单个样本记忆性的随机性。实验结果表明，我们的方法在不同数据分布和规模下均显著优于先前方法。此外，我们的方法对早期停止和数据增强等过拟合缓解策略表现出优越的抵抗能力。