Differential privacy (DP) offers a theoretical upper bound on the potential privacy leakage of analgorithm, while empirical auditing establishes a practical lower bound. Auditing techniques exist forDP training algorithms. However machine learning can also be made private at inference. We propose thefirst framework for auditing private prediction where we instantiate adversaries with varying poisoningand query capabilities. This enables us to study the privacy leakage of four private prediction algorithms:PATE [Papernot et al., 2016], CaPC [Choquette-Choo et al., 2020], PromptPATE [Duan et al., 2023],and Private-kNN [Zhu et al., 2020]. To conduct our audit, we introduce novel techniques to empiricallyevaluate privacy leakage in terms of Renyi DP. Our experiments show that (i) the privacy analysis ofprivate prediction can be improved, (ii) algorithms which are easier to poison lead to much higher privacyleakage, and (iii) the privacy leakage is significantly lower for adversaries without query control than thosewith full control.

翻译：差分隐私（DP）为算法的潜在隐私泄露提供了理论上的上界，而经验审计则确立了实际的下界。针对DP训练算法的审计技术已有研究，然而机器学习亦可在推理阶段实现隐私保护。我们提出了首个用于审计私有预测的框架，在此框架中，我们实例化了具有不同投毒和查询能力的攻击者。这使我们能够研究四种私有预测算法的隐私泄露情况：PATE [Papernot 等人，2016]、CaPC [Choquette-Choo 等人，2020]、PromptPATE [Duan 等人，2023] 以及 Private-kNN [Zhu 等人，2020]。为执行审计，我们引入了新技术，以基于Renyi DP经验性地评估隐私泄露。实验表明：（i）私有预测的隐私分析仍有改进空间，（ii）易于投毒的算法会导致更高的隐私泄露，（iii）与具有完全控制权的攻击者相比，无查询控制权的攻击者所引发的隐私泄露显著更低。