Individual Differential Privacy (iDP) promises users control over their privacy, but this promise can be broken in practice. We reveal a previously overlooked vulnerability in sampling-based iDP mechanisms: while conforming to the iDP guarantees, an individual's privacy risk is not solely governed by their own privacy budget, but critically depends on the privacy choices of all other data contributors. This creates a mismatch between the promise of individual privacy control and the reality of a system where risk is collectively determined. We demonstrate empirically that certain distributions of privacy preferences can unintentionally inflate the privacy risk of individuals, even when their formal guarantees are met. Moreover, this excess risk provides an exploitable attack vector. A central adversary or a set of colluding adversaries can deliberately choose privacy budgets to amplify vulnerabilities of targeted individuals. Most importantly, this attack operates entirely within the guarantees of DP, hiding this excess vulnerability. Our empirical evaluation demonstrates successful attacks against 62% of targeted individuals, substantially increasing their membership inference susceptibility. To mitigate this, we propose $(\varepsilon_i,δ_i,\overlineΔ)$-iDP a privacy contract that uses $Δ$-divergences to provide users with a hard upper bound on their excess vulnerability, while offering flexibility to mechanism design. Our findings expose a fundamental challenge to the current paradigm, demanding a re-evaluation of how iDP systems are designed, audited, communicated, and deployed to make excess risks transparent and controllable.
翻译:个体差分隐私(iDP)承诺用户能控制其隐私,但这一承诺在实践中可能被打破。我们揭示了基于采样的iDP机制中一个先前被忽视的漏洞:尽管符合iDP保证,个体的隐私风险不仅由其自身的隐私预算决定,还关键取决于所有其他数据贡献者的隐私选择。这导致了个人隐私控制的承诺与风险由集体决定的系统现实之间的错配。我们通过实证证明,某些隐私偏好的分布可能无意中放大个体的隐私风险,即使其形式化保证已得到满足。此外,这种额外风险提供了一个可利用的攻击向量。一个中心对手或一组合谋对手可以故意选择隐私预算,以放大目标个体的脆弱性。最重要的是,这种攻击完全在DP的保证范围内运作,从而隐藏了这种额外的脆弱性。我们的实证评估表明,针对62%的目标个体的攻击是成功的,显著增加了其成员推理的易感性。为缓解此问题,我们提出了$(\varepsilon_i,\delta_i,\overline{\Delta})$-iDP隐私合约,该合约利用$\Delta$-散度为用户提供其额外脆弱性的严格上界,同时为机制设计提供灵活性。我们的研究结果揭示了当前范式面临的一个根本性挑战,要求重新评估iDP系统的设计、审计、沟通和部署方式,以使额外风险透明且可控。