The Probably Approximately Correct (PAC) Privacy framework [46] provides a powerful instance-based methodology to preserve privacy in complex data-driven systems. Existing PAC Privacy algorithms (we call them Auto-PAC) rely on a Gaussian mutual information upper bound. However, we show that the upper bound obtained by these algorithms is tight if and only if the perturbed mechanism output is jointly Gaussian with independent Gaussian noise. We propose two approaches for addressing this issue. First, we introduce two tractable post-processing methods for Auto-PAC, based on Donsker-Varadhan representation and sliced Wasserstein distances. However, the result still leaves wasted privacy budget. To address this issue more fundamentally, we introduce Residual-PAC (R-PAC) Privacy, an f-divergence-based measure to quantify privacy that remains after adversarial inference. To implement R-PAC Privacy in practice, we propose a Stackelberg Residual-PAC (SR-PAC) privatization mechanism, a game-theoretic framework that selects optimal noise distributions through convex bilevel optimization. Our approach achieves efficient privacy budget utilization for arbitrary data distributions and naturally composes when multiple mechanisms access the dataset. Through extensive experiments, we demonstrate that SR-PAC consistently obtains a better privacy-utility tradeoff than both PAC and differential privacy baselines.

翻译：概率近似正确（PAC）隐私框架[46]提供了一种强大的基于实例的方法论，用于保护复杂数据驱动系统中的隐私。现有的PAC隐私算法（我们称之为Auto-PAC）依赖于高斯互信息上界。然而，我们证明这些算法获得的上界是紧的当且仅当扰动机制输出与独立高斯噪声联合服从高斯分布。我们提出了两种解决此问题的方法。首先，我们基于Donsker-Varadhan表示和切片Wasserstein距离，为Auto-PAC引入了两种易处理的后处理方法。然而，其结果仍会浪费隐私预算。为更根本地解决此问题，我们提出了残差PAC（R-PAC）隐私——一种基于f-散度的度量方法，用于量化对抗性推断后剩余的隐私。为实现R-PAC隐私的实际应用，我们提出了Stackelberg残差PAC（SR-PAC）私有化机制，这是一个通过凸双层优化选择最优噪声分布的博弈论框架。我们的方法能针对任意数据分布实现高效的隐私预算利用，并在多个机制访问数据集时自然组合。通过大量实验，我们证明SR-PAC在隐私-效用权衡方面始终优于PAC和差分隐私基线。