Reconstruction attacks and defenses are essential in understanding the data leakage problem in machine learning. However, prior work has centered around empirical observations of gradient inversion attacks, lacks theoretical grounding, and cannot disentangle the usefulness of defending methods from the computational limitation of attacking methods. In this work, we propose to view the problem as an inverse problem, enabling us to theoretically and systematically evaluate the data reconstruction attack. On various defense methods, we derived the algorithmic upper bound and the matching (in feature dimension and architecture dimension) information-theoretical lower bound on the reconstruction error for two-layer neural networks. To complement the theoretical results and investigate the utility-privacy trade-off, we defined a natural evaluation metric of the defense methods with similar utility loss among the strongest attacks. We further propose a strong reconstruction attack that helps update some previous understanding of the strength of defense methods under our proposed evaluation metric.

翻译：重构攻击与防御对于理解机器学习中的数据泄露问题至关重要。然而，先前的研究主要集中于梯度反演攻击的经验性观察，缺乏理论基础，且无法区分防御方法的有效性与攻击方法计算局限性的影响。在本研究中，我们提出将该问题视为逆问题，从而能够在理论上系统性地评估数据重构攻击。针对多种防御方法，我们推导出了双层神经网络重构误差的算法上界，以及与之匹配（在特征维度和架构维度上）的信息论下界。为补充理论结果并探究效用-隐私权衡关系，我们定义了在最强攻击下具有相似效用损失的防御方法自然评估指标。我们进一步提出了一种强重构攻击方法，该方法有助于更新先前关于防御方法在我们所提评估指标下强度的认知。