Differentially private training offers a protection which is usually interpreted as a guarantee against membership inference attacks. By proxy, this guarantee extends to other threats like reconstruction attacks attempting to extract complete training examples. Recent works provide evidence that if one does not need to protect against membership attacks but instead only wants to protect against training data reconstruction, then utility of private models can be improved because less noise is required to protect against these more ambitious attacks. We investigate this further in the context of DP-SGD, a standard algorithm for private deep learning, and provide an upper bound on the success of any reconstruction attack against DP-SGD together with an attack that empirically matches the predictions of our bound. Together, these two results open the door to fine-grained investigations on how to set the privacy parameters of DP-SGD in practice to protect against reconstruction attacks. Finally, we use our methods to demonstrate that different settings of the DP-SGD parameters leading to the same DP guarantees can result in significantly different success rates for reconstruction, indicating that the DP guarantee alone might not be a good proxy for controlling the protection against reconstruction attacks.

翻译：差分隐私训练提供了一种保护，通常被解释为对成员推断攻击的保障。作为代理，这种保障也扩展到其他威胁，如试图提取完整训练样本的重构攻击。近期研究表明，若无需防御成员攻击而仅需防御训练数据重构，则可提高私有模型的效用，因为防范这些更具野心的攻击所需噪声更少。我们在DP-SGD（私有深度学习标准算法）背景下进一步探究，为针对DP-SGD的任何重构攻击的成功概率提供了上界，并提出了一个与边界预测经验匹配的攻击方法。这两项结果共同为实践中如何设置DP-SGD隐私参数以防御重构攻击的精细研究打开了大门。最后，我们利用所提方法证明：导致相同DP保障的DP-SGD参数的不同设置，可能对应显著不同的重构成功率，这表明仅凭DP保障本身可能并非控制重构攻击防护的良好代理指标。