Membership inference attacks (MIAs) are widely used to empirically assess the privacy risks of samples used to train a target machine learning model. State-of-the-art methods however require training hundreds of shadow models, with the same size and architecture of the target model, solely to evaluate the privacy risk. While one might be able to afford this for small models, the cost often becomes prohibitive for medium and large models. We here instead propose a novel approach to identify the at-risk samples using only artifacts available during training, with little to no additional computational overhead. Our method analyzes individual per-sample loss traces and uses them to identify the vulnerable data samples. We demonstrate the effectiveness of our artifact-based approach through experiments on the CIFAR10 dataset, showing high precision in identifying vulnerable samples as determined by a SOTA shadow model-based MIA (LiRA). Impressively, our method reaches the same precision as another SOTA MIA when measured against LiRA, despite it being orders of magnitude cheaper. We then show LT-IQR to outperform alternative loss aggregation methods, perform ablation studies on hyperparameters, and validate the robustness of our method to the target metric. Finally, we study the evolution of the vulnerability score distribution throughout training as a metric for model-level risk assessment.

翻译：成员推理攻击被广泛用于实证评估用于训练目标机器学习模型的样本的隐私风险。然而，现有最先进的方法需要训练数百个与目标模型规模及架构相同的影子模型，仅用于评估隐私风险。虽然对于小型模型尚可承受此成本，但对于中型和大型模型，其开销往往变得难以负担。本文提出一种仅利用训练过程中可获得的产物来识别风险样本的新方法，该方法几乎不产生额外计算开销。我们的方法通过分析单个样本的损失轨迹，并利用这些轨迹识别易受攻击的数据样本。我们在CIFAR10数据集上通过实验证明了基于产物的方法的有效性，其在识别易受攻击样本方面表现出高精度，该精度由基于影子模型的最先进成员推理攻击所确定。值得注意的是，尽管我们的方法计算成本低数个数量级，但在以LiRA为基准进行评估时，其精度与另一种最先进成员推理攻击相当。我们进一步证明LT-IQR方法优于其他损失聚合方法，对超参数进行消融研究，并验证了方法对目标指标的鲁棒性。最后，我们通过研究训练过程中脆弱性评分分布的演变，将其作为模型级风险评估的度量指标。