Machine learning models, in particular deep neural networks, are currently an integral part of various applications, from healthcare to finance. However, using sensitive data to train these models raises concerns about privacy and security. One method that has emerged to verify if the trained models are privacy-preserving is Membership Inference Attacks (MIA), which allows adversaries to determine whether a specific data point was part of a model's training dataset. While a series of MIAs have been proposed in the literature, only a few can achieve high True Positive Rates (TPR) in the low False Positive Rate (FPR) region (0.01%~1%). This is a crucial factor to consider for an MIA to be practically useful in real-world settings. In this paper, we present a novel approach to MIA that is aimed at significantly improving TPR at low FPRs. Our method, named learning-based difficulty calibration for MIA(LDC-MIA), characterizes data records by their hardness levels using a neural network classifier to determine membership. The experiment results show that LDC-MIA can improve TPR at low FPR by up to 4x compared to the other difficulty calibration based MIAs. It also has the highest Area Under ROC curve (AUC) across all datasets. Our method's cost is comparable with most of the existing MIAs, but is orders of magnitude more efficient than one of the state-of-the-art methods, LiRA, while achieving similar performance.

翻译：机器学习模型，特别是深度神经网络，目前已广泛应用于从医疗到金融的各个领域。然而，使用敏感数据训练这些模型引发了隐私与安全方面的担忧。为验证训练模型是否具有隐私保护能力，成员推断攻击作为一种方法应运而生，它允许攻击者判断特定数据点是否属于模型训练数据集。尽管文献中已提出一系列成员推断攻击方法，但仅少数能在低假阳性率（0.01%~1%）区域实现高真阳性率。这对于成员推断攻击在实际场景中的实用价值至关重要。本文提出一种针对成员推断攻击的新型方法，旨在显著提升低假阳性率下的真阳性率。我们提出的方法——基于学习的难度校准成员推断攻击，通过神经网络分类器根据数据的难度等级对记录进行表征，从而判定成员关系。实验结果表明，与其他基于难度校准的成员推断攻击相比，LDC-MIA在低假阳性率下的真阳性率提升可达4倍。同时，它在所有数据集上均获得最高的ROC曲线下面积值。本方法的计算成本与现有大多数成员推断攻击相当，但相较于当前最先进方法之一LiRA，其效率提升了数个数量级，且性能表现相近。