Bounding privacy leakage over compositions, i.e., privacy accounting, is a key challenge in differential privacy (DP). However, the privacy parameter ($\varepsilon$ or $\delta$) is often easy to estimate but hard to bound. In this paper, we propose a new differential privacy paradigm called estimate-verify-release (EVR), which addresses the challenges of providing a strict upper bound for privacy parameter in DP compositions by converting an estimate of privacy parameter into a formal guarantee. The EVR paradigm first estimates the privacy parameter of a mechanism, then verifies whether it meets this guarantee, and finally releases the query output based on the verification result. The core component of the EVR is privacy verification. We develop a randomized privacy verifier using Monte Carlo (MC) technique. Furthermore, we propose an MC-based DP accountant that outperforms existing DP accounting techniques in terms of accuracy and efficiency. Our empirical evaluation shows the newly proposed EVR paradigm improves the utility-privacy tradeoff for privacy-preserving machine learning.

翻译：在差分隐私中，对组合隐私泄露进行界限设定（即隐私核算）是一项关键挑战。然而，隐私参数（$\varepsilon$ 或 $\delta$）通常易于估计但难以界定。本文提出一种名为“估计-验证-发布”（EVR）的新型差分隐私范式，通过将隐私参数的估计转化为形式化保证，解决了在差分隐私组合中为隐私参数提供严格上界的难题。EVR 范式首先估计机制的隐私参数，接着验证该参数是否满足保证，最后根据验证结果释放查询输出。其核心组件是隐私验证。我们利用蒙特卡洛（MC）技术开发了一种随机隐私验证器。此外，我们提出了一种基于 MC 的隐私核算方法，在准确性和效率上均优于现有差分隐私核算技术。实验评估表明，新提出的 EVR 范式改善了隐私保护机器学习中的效用-隐私权衡。