Membership inference attacks are designed to determine, using black box access to trained models, whether a particular example was used in training or not. Membership inference can be formalized as a hypothesis testing problem. The most effective existing attacks estimate the distribution of some test statistic (usually the model's confidence on the true label) on points that were (and were not) used in training by training many \emph{shadow models} -- i.e. models of the same architecture as the model being attacked, trained on a random subsample of data. While effective, these attacks are extremely computationally expensive, especially when the model under attack is large. We introduce a new class of attacks based on performing quantile regression on the distribution of confidence scores induced by the model under attack on points that are not used in training. We show that our method is competitive with state-of-the-art shadow model attacks, while requiring substantially less compute because our attack requires training only a single model. Moreover, unlike shadow model attacks, our proposed attack does not require any knowledge of the architecture of the model under attack and is therefore truly ``black-box". We show the efficacy of this approach in an extensive series of experiments on various datasets and model architectures.

翻译：成员推断攻击旨在通过黑盒访问已训练模型，判断特定样本是否用于训练过程。该问题可形式化为假设检验问题。现有最有效的攻击方法通过训练多个影子模型——即与被攻击模型架构相同、在随机数据子集上训练的模型——来估计样本在训练集内外的某种检验统计量（通常是模型对真实标签的置信度）分布。尽管有效，此类攻击计算成本极高，尤其当被攻击模型规模较大时。本文提出一类基于分位数回归的新攻击方法，通过分析被攻击模型在非训练样本上产生的置信度分数分布来实现推断。研究表明，本方法在保持与最先进的影子模型攻击相当性能的同时，由于仅需训练单个模型，计算量显著降低。更重要的是，与影子模型攻击不同，本方法无需知晓被攻击模型的架构信息，因此是真正的"黑盒"攻击。我们在多种数据集和模型架构上开展了一系列广泛实验，验证了该方法的有效性。