Text-to-image diffusion models have achieved tremendous success in the field of controllable image generation, while also coming along with issues of privacy leakage and data copyrights. Membership inference arises in these contexts as a potential auditing method for detecting unauthorized data usage. While some efforts have been made on diffusion models, they are not applicable to text-to-image diffusion models due to the high computation overhead and enhanced generalization capabilities. In this paper, we first identify a conditional overfitting phenomenon in text-to-image diffusion models, indicating that these models tend to overfit the conditional distribution of images given the corresponding text rather than the marginal distribution of images only. Based on this observation, we derive an analytical indicator, namely Conditional Likelihood Discrepancy (CLiD), to perform membership inference, which reduces the stochasticity in estimating memorization of individual samples. Experimental results demonstrate that our method significantly outperforms previous methods across various data distributions and dataset scales. Additionally, our method shows superior resistance to overfitting mitigation strategies, such as early stopping and data augmentation.

翻译：文本到图像扩散模型在可控图像生成领域取得了巨大成功，同时也伴随着隐私泄露和数据版权问题。在此背景下，成员推断作为一种检测未经授权数据使用的潜在审计方法应运而生。尽管已有一些针对扩散模型的研究工作，但由于高计算开销和增强的泛化能力，这些方法并不适用于文本到图像扩散模型。本文首先识别了文本到图像扩散模型中的条件过拟合现象，表明这些模型倾向于过拟合给定对应文本的图像条件分布，而非仅图像的边缘分布。基于这一观察，我们推导出一个分析性指标——条件似然差异（CLiD），用于执行成员推断，从而降低估计单个样本记忆性的随机性。实验结果表明，我们的方法在各种数据分布和数据集规模上均显著优于先前方法。此外，我们的方法对过拟合缓解策略（如早停法和数据增强）表现出卓越的抵抗能力。