Posterior sampling, i.e., exponential mechanism to sample from the posterior distribution, provides $\varepsilon$-pure differential privacy (DP) guarantees and does not suffer from potentially unbounded privacy breach introduced by $(\varepsilon,\delta)$-approximate DP. In practice, however, one needs to apply approximate sampling methods such as Markov chain Monte Carlo (MCMC), thus re-introducing the unappealing $\delta$-approximation error into the privacy guarantees. To bridge this gap, we propose the Approximate SAample Perturbation (abbr. ASAP) algorithm which perturbs an MCMC sample with noise proportional to its Wasserstein-infinity ($W_\infty$) distance from a reference distribution that satisfies pure DP or pure Gaussian DP (i.e., $\delta=0$). We then leverage a Metropolis-Hastings algorithm to generate the sample and prove that the algorithm converges in W$_\infty$ distance. We show that by combining our new techniques with a careful localization step, we obtain the first nearly linear-time algorithm that achieves the optimal rates in the DP-ERM problem with strongly convex and smooth losses.

翻译：后验采样（即从后验分布中采样的指数机制）可提供ε-纯差分隐私（DP）保证，且不会遭受由(ε,δ)-近似DP引入的潜在无界隐私泄露问题。然而在实践中，需要应用马尔可夫链蒙特卡洛（MCMC）等近似采样方法，这导致不可容忍的δ近似误差重新被引入隐私保证中。为解决这一矛盾，我们提出近似样本扰动（简称ASAP）算法，该算法通过将MCMC样本与参考分布的Wasserstein无穷范数（W∞）距离成比例的噪声进行扰动，其中参考分布满足纯DP或纯高斯DP（即δ=0）。随后利用Metropolis-Hastings算法生成样本，并证明该算法在W∞距离下收敛。通过将新技术与精细的局部化步骤相结合，我们首次在强凸光滑损失函数的DP-ERM问题中，获得了达到最优速率的近乎线性时间算法。