In differentially private (DP) machine learning, the privacy guarantees of DP mechanisms are often reported and compared on the basis of a single $(\varepsilon, \delta)$-pair. This practice overlooks that DP guarantees can vary substantially even between mechanisms sharing a given $(\varepsilon, \delta)$, and potentially introduces privacy vulnerabilities which can remain undetected. This motivates the need for robust, rigorous methods for comparing DP guarantees in such cases. Here, we introduce the $\Delta$-divergence between mechanisms which quantifies the worst-case excess privacy vulnerability of choosing one mechanism over another in terms of $(\varepsilon, \delta)$, $f$-DP and in terms of a newly presented Bayesian interpretation. Moreover, as a generalisation of the Blackwell theorem, it is endowed with strong decision-theoretic foundations. Through application examples, we show that our techniques can facilitate informed decision-making and reveal gaps in the current understanding of privacy risks, as current practices in DP-SGD often result in choosing mechanisms with high excess privacy vulnerabilities.

翻译：在差分隐私（DP）机器学习中，DP机制的隐私保障通常基于单一的$(\varepsilon, \delta)$参数对进行报告和比较。这种做法忽略了即使共享给定$(\varepsilon, \delta)$的机制之间，DP保障也可能存在显著差异，并可能引入未被察觉的隐私漏洞。这促使我们需要在此类情况下进行稳健、严谨的DP保障比较方法。本文引入机制间的$\Delta$-散度，该度量从$(\varepsilon, \delta)$、$f$-DP以及新提出的贝叶斯解释角度，量化了选择一种机制而非另一种时所产生的最坏情况超额隐私风险。此外，作为布莱克韦尔定理的推广，该度量具备坚实的决策理论基础。通过应用示例，我们证明所提技术能够促进知情决策，并揭示当前对隐私风险认知的不足——当前DP-SGD的实践往往导致选择具有高超额隐私风险的机制。