Membership Inference Attacks (MIA) aim to infer whether a target data record has been utilized for model training or not. Prior attempts have quantified the privacy risks of language models (LMs) via MIAs, but there is still no consensus on whether existing MIA algorithms can cause remarkable privacy leakage on practical Large Language Models (LLMs). Existing MIAs designed for LMs can be classified into two categories: reference-free and reference-based attacks. They are both based on the hypothesis that training records consistently strike a higher probability of being sampled. Nevertheless, this hypothesis heavily relies on the overfitting of target models, which will be mitigated by multiple regularization methods and the generalization of LLMs. The reference-based attack seems to achieve promising effectiveness in LLMs, which measures a more reliable membership signal by comparing the probability discrepancy between the target model and the reference model. However, the performance of reference-based attack is highly dependent on a reference dataset that closely resembles the training dataset, which is usually inaccessible in the practical scenario. Overall, existing MIAs are unable to effectively unveil privacy leakage over practical fine-tuned LLMs that are overfitting-free and private. We propose a Membership Inference Attack based on Self-calibrated Probabilistic Variation (SPV-MIA). Specifically, since memorization in LLMs is inevitable during the training process and occurs before overfitting, we introduce a more reliable membership signal, probabilistic variation, which is based on memorization rather than overfitting. Furthermore, we introduce a self-prompt approach, which constructs the dataset to fine-tune the reference model by prompting the target LLM itself. In this manner, the adversary can collect a dataset with a similar distribution from public APIs.

翻译：成员推断攻击旨在推断目标数据记录是否被用于模型训练。先前研究通过成员推断攻击量化了语言模型的隐私风险，但现有攻击算法能否对实用大型语言模型造成显著隐私泄露尚无定论。现有面向语言模型的成员推断攻击可分为无参考与基于参考两类，两者均基于训练记录始终具有更高采样概率的假设。然而，该假设高度依赖目标模型的过拟合现象，而正则化方法与大型语言模型的泛化能力会缓解此现象。基于参考的攻击在大型语言模型中展现出潜力：通过比较目标模型与参考模型的概率差异，可获取更可靠的成员信号。但此类攻击性能高度依赖与训练集分布相似的参考数据集，而实际场景中通常无法获取该数据集。总体而言，现有成员推断攻击无法有效揭示无过拟合的私有微调大型语言模型中的隐私泄露。本文提出基于自校准概率变异的成员推断攻击。具体而言，由于大型语言模型训练过程中记忆现象不可避免且先于过拟合发生，我们引入基于记忆而非过拟合的更可靠成员信号——概率变异。同时，我们提出自提示方法，通过提示目标大型语言模型自身来构建数据集并微调参考模型。通过该方法，攻击者可从公共API收集分布相似的参考数据集。