Modern machine learning (ML) ecosystems offer a surging number of ML frameworks and code repositories that can greatly facilitate the development of ML models. Today, even ordinary data holders who are not ML experts can apply off-the-shelf codebase to build high-performance ML models on their data, many of which are sensitive in nature (e.g., clinical records). In this work, we consider a malicious ML provider who supplies model-training code to the data holders, does not have access to the training process, and has only black-box query access to the resulting model. In this setting, we demonstrate a new form of membership inference attack that is strictly more powerful than prior art. Our attack empowers the adversary to reliably de-identify all the training samples (average >99% attack [email protected]% FPR), and the compromised models still maintain competitive performance as their uncorrupted counterparts (average <1% accuracy drop). Moreover, we show that the poisoned models can effectively disguise the amplified membership leakage under common membership privacy auditing, which can only be revealed by a set of secret samples known by the adversary. Overall, our study not only points to the worst-case membership privacy leakage, but also unveils a common pitfall underlying existing privacy auditing methods, which calls for future efforts to rethink the current practice of auditing membership privacy in machine learning models.

翻译：现代机器学习生态系统提供了数量激增的机器学习框架和代码仓库，这极大地便利了机器学习模型的开发。如今，即使是并非机器学习专家的普通数据持有者，也能应用现成的代码库在其数据上构建高性能的机器学习模型，其中许多数据本质上是敏感的（例如临床记录）。在本工作中，我们考虑一个恶意的机器学习服务提供商，其向数据持有者提供模型训练代码，无法访问训练过程，并且仅对最终模型拥有黑盒查询访问权限。在此设定下，我们展示了一种严格优于现有技术的新型成员推理攻击。我们的攻击使对手能够可靠地识别出所有训练样本（平均攻击真阳性率>99%@0.1%假阳性率），而被攻陷的模型仍能保持与未受污染模型相竞争的性能（平均准确率下降<1%）。此外，我们表明，中毒模型能有效掩盖在常见成员隐私审计下被放大的成员信息泄露，这种泄露只能通过一组对手已知的秘密样本被揭示。总体而言，我们的研究不仅指出了最坏情况下的成员隐私泄露，而且揭示了现有隐私审计方法背后的一个常见缺陷，这呼吁未来重新思考当前机器学习模型中成员隐私审计的实践。