Differentially private training offers a protection which is usually interpreted as a guarantee against membership inference attacks. By proxy, this guarantee extends to other threats like reconstruction attacks attempting to extract complete training examples. Recent works provide evidence that if one does not need to protect against membership attacks but instead only wants to protect against training data reconstruction, then utility of private models can be improved because less noise is required to protect against these more ambitious attacks. We investigate this further in the context of DP-SGD, a standard algorithm for private deep learning, and provide an upper bound on the success of any reconstruction attack against DP-SGD together with an attack that empirically matches the predictions of our bound. Together, these two results open the door to fine-grained investigations on how to set the privacy parameters of DP-SGD in practice to protect against reconstruction attacks. Finally, we use our methods to demonstrate that different settings of the DP-SGD parameters leading to the same DP guarantees can result in significantly different success rates for reconstruction, indicating that the DP guarantee alone might not be a good proxy for controlling the protection against reconstruction attacks.

翻译：差分隐私训练提供了一种通常被解读为防御成员推断攻击的保障。通过这一代理机制，该保障可扩展至其他威胁，例如试图提取完整训练样本的重构攻击。近期研究证明，若无需防御成员攻击而仅需防范训练数据重构，则可减少所需噪声量以提升隐私模型的效用，因为此类更具挑战性的攻击需要更少的保护强度。我们以差分隐私随机梯度下降（DP-SGD）这一标准私有深度学习算法为背景展开深入研究，提出了针对DP-SGD重构攻击成功概率的上界，并设计了一种与理论边界预测相匹配的实证攻击方法。这两项成果共同为实践中如何精细调控DP-SGD的隐私参数以防御重构攻击提供了研究路径。最后，我们通过实验表明：在保持相同差分隐私保证的前提下，DP-SGD参数的不同配置会导致显著差异化的重构成功率，这证实了仅依赖差分隐私保证可能并非控制重构攻击防护效果的可靠代理指标。