Membership inference attacks (MIAs) aim to determine whether a data sample was included in a machine learning (ML) model's training set and have become the de facto standard for measuring privacy leakages in ML. We propose an evaluation framework that defines the conditions under which MIAs constitute a genuine privacy threat, and review representative MIAs against it. We find that, under the realistic conditions defined in our framework, MIAs represent weak privacy threats. Thus, relying on them as a privacy metric in ML can lead to an overestimation of risk and to unnecessary sacrifices in model utility as a consequence of employing too strong defenses.

翻译：成员推断攻击（MIAs）旨在判定某数据样本是否包含在机器学习（ML）模型的训练集中，并已成为衡量ML隐私泄露的事实标准。我们提出一个评估框架，定义了MIAs构成真正隐私威胁的条件，并据此对代表性MIAs进行了评述。研究发现，在框架所定义的实际条件下，MIAs仅构成较弱的隐私威胁。因此，将其作为ML隐私度量可能导致风险高估，并因采用过强防御而牺牲不必要的模型效用。