Training machine learning models on privacy-sensitive data has become a popular practice, driving innovation in ever-expanding fields. This has opened the door to new attacks that can have serious privacy implications. One such attack, the Membership Inference Attack (MIA), exposes whether or not a particular data point was used to train a model. A growing body of literature uses Differentially Private (DP) training algorithms as a defence against such attacks. However, these works evaluate the defence under the restrictive assumption that all members of the training set, as well as non-members, are independent and identically distributed. This assumption does not hold for many real-world use cases in the literature. Motivated by this, we evaluate membership inference with statistical dependencies among samples and explain why DP does not provide meaningful protection (the privacy parameter $\epsilon$ scales with the training set size $n$) in this more general case. We conduct a series of empirical evaluations with off-the-shelf MIAs using training sets built from real-world data showing different types of dependencies among samples. Our results reveal that training set dependencies can severely increase the performance of MIAs, and therefore assuming that data samples are statistically independent can significantly underestimate the performance of MIAs.

翻译：在隐私敏感数据上训练机器学习模型已成为普遍做法，推动了不断扩展领域中的创新。这一趋势也催生了可能带来严重隐私风险的新型攻击。成员推断攻击（MIA）便是其中之一，它能揭露某个特定数据点是否被用于模型训练。越来越多文献采用差分隐私（DP）训练算法作为抵御此类攻击的手段。然而，这些研究在评估防御效果时均基于一个严格假设：训练集所有成员及非成员样本独立同分布。这一假设与许多文献中的现实应用场景不符。基于此，我们评估了样本存在统计依赖关系时的成员推断问题，并解释了为何在此更普遍的情况下DP无法提供有意义的保护（隐私参数$\epsilon$随训练集大小$n$呈线性增长）。我们使用从现实世界数据构建、展现不同类型样本依赖关系的训练集，通过现成的MIA方法开展了一系列实证评估。结果表明，训练集依赖关系会显著提升MIA的攻击性能，因此假设数据样本统计独立会严重低估MIA的实际表现。