Machine learning models, in particular deep neural networks, are currently an integral part of various applications, from healthcare to finance. However, using sensitive data to train these models raises concerns about privacy and security. One method that has emerged to verify if the trained models are privacy-preserving is Membership Inference Attacks (MIA), which allows adversaries to determine whether a specific data point was part of a model's training dataset. While a series of MIAs have been proposed in the literature, only a few can achieve high True Positive Rates (TPR) in the low False Positive Rate (FPR) region (0.01%~1%). This is a crucial factor to consider for an MIA to be practically useful in real-world settings. In this paper, we present a novel approach to MIA that is aimed at significantly improving TPR at low FPRs. Our method, named learning-based difficulty calibration for MIA(LDC-MIA), characterizes data records by their hardness levels using a neural network classifier to determine membership. The experiment results show that LDC-MIA can improve TPR at low FPR by up to 4x compared to the other difficulty calibration based MIAs. It also has the highest Area Under ROC curve (AUC) across all datasets. Our method's cost is comparable with most of the existing MIAs, but is orders of magnitude more efficient than one of the state-of-the-art methods, LiRA, while achieving similar performance.

翻译：机器学习模型，尤其是深度神经网络，当前已成为从医疗到金融等各类应用中不可或缺的一部分。然而，使用敏感数据训练这些模型引发了对隐私和安全的担忧。成员推断攻击（MIA）作为一种验证训练模型是否保护隐私的方法而出现，它使攻击者能够确定某个特定数据点是否属于模型训练数据集。尽管文献中提出了一系列成员推断攻击，但在低误报率（FPR）区域（0.01%~1%），仅有少数方法能实现高真阳性率（TPR）。这对于成员推断攻击在实际场景中具有实用价值是一个关键考量因素。本文提出了一种新颖的成员推断攻击方法，旨在显著提升低误报率下的真阳性率。我们的方法名为基于学习的难度校准成员推断攻击（LDC-MIA），通过使用神经网络分类器根据数据记录的难度级别来判定其成员资格。实验结果表明，与基于难度校准的其他成员推断攻击相比，LDC-MIA在低误报率下的真阳性率可提升高达4倍。同时，它还在所有数据集上取得了最高的ROC曲线下面积（AUC）。我们的方法成本与大多数现有成员推断攻击相当，但比最先进方法之一LiRA高效数个数量级，同时实现了与之相似的性能。