Recent methods for auditing the privacy of machine learning algorithms have improved computational efficiency by simultaneously intervening on multiple training examples in a single training run. Steinke et al. (2024) prove that one-run auditing indeed lower bounds the true privacy parameter of the audited algorithm, and give impressive empirical results. Their work leaves open the question of how precisely one-run auditing can uncover the true privacy parameter of an algorithm, and how that precision depends on the audited algorithm. In this work, we characterize the maximum achievable efficacy of one-run auditing and show that one-run auditing can only perfectly uncover the true privacy parameters of algorithms whose structure allows the effects of individual data elements to be isolated. Our characterization helps reveal how and when one-run auditing is still a promising technique for auditing real machine learning algorithms, despite these fundamental gaps.

翻译：近期用于审计机器学习算法隐私性的方法通过单次训练运行中同时对多个训练样本进行干预，提高了计算效率。Steinke等人（2024）证明单次运行审计确实能够给出被审计算法真实隐私参数的下界，并展示了令人印象深刻的实证结果。他们的研究遗留了一个问题：单次运行审计能在多大程度上精确揭示算法的真实隐私参数，以及这种精度如何依赖于被审计算法。在本研究中，我们刻画了单次运行审计可达到的最大效能，并证明单次运行审计仅能完美揭示那些结构允许隔离单个数据元素影响的算法的真实隐私参数。我们的刻画有助于揭示，尽管存在这些根本性局限，单次运行审计在何种情况下以及如何仍是一种有前景的审计实际机器学习算法的技术。