Membership Inference attacks (MIAs) aim to predict whether a data sample was present in the training data of a machine learning model or not, and are widely used for assessing the privacy risks of language models. Most existing attacks rely on the observation that models tend to assign higher probabilities to their training samples than non-training points. However, simple thresholding of the model score in isolation tends to lead to high false-positive rates as it does not account for the intrinsic complexity of a sample. Recent work has demonstrated that reference-based attacks which compare model scores to those obtained from a reference model trained on similar data can substantially improve the performance of MIAs. However, in order to train reference models, attacks of this kind make the strong and arguably unrealistic assumption that an adversary has access to samples closely resembling the original training data. Therefore, we investigate their performance in more realistic scenarios and find that they are highly fragile in relation to the data distribution used to train reference models. To investigate whether this fragility provides a layer of safety, we propose and evaluate neighbourhood attacks, which compare model scores for a given sample to scores of synthetically generated neighbour texts and therefore eliminate the need for access to the training data distribution. We show that, in addition to being competitive with reference-based attacks that have perfect knowledge about the training data distribution, our attack clearly outperforms existing reference-free attacks as well as reference-based attacks with imperfect knowledge, which demonstrates the need for a reevaluation of the threat model of adversarial attacks.

翻译：成员推理攻击旨在判断数据样本是否存在于机器学习模型的训练数据中，被广泛用于评估语言模型的隐私风险。现有大多数攻击基于一个观察：模型分配给训练样本的概率往往高于非训练样本。然而，对模型得分进行简单阈值处理容易导致高假阳性率，因其未能考虑样本的内在复杂度。近期研究证明，基于参考的攻击——即将模型得分与在相似数据上训练的参考模型得分进行比较——能显著提升成员推理攻击的性能。但此类攻击需要假设攻击者能获取与原始训练数据高度相似的样本，这一假设过于严格且不切实际。因此，我们研究了这类攻击在更真实场景下的表现，发现其对训练参考模型所用的数据分布高度敏感。为探究这种脆弱性是否构成安全层，我们提出并评估了邻域攻击——该方法将给定样本的模型得分与合成生成的邻域文本得分进行比较，从而无需获取训练数据分布。实验表明，我们的攻击不仅能与完全掌握训练数据分布的基于参考的攻击相抗衡，还显著优于现有的无参考攻击及知识不完整的基于参考的攻击，这证明需要重新评估对抗攻击的威胁模型。