Federated Learning has seen an increased deployment in real-world scenarios recently, as it enables the distributed training of machine learning models without explicit data sharing between individual clients. Yet, the introduction of the so-called gradient inversion attacks has fundamentally challenged its privacy-preserving properties. Unfortunately, as these attacks mostly rely on direct data optimization without any formal guarantees, the vulnerability of real-world systems remains in dispute and requires tedious testing for each new federated deployment. To overcome these issues, recently the SPEAR attack was introduced, which is based on a theoretical analysis of the gradients of linear layers with ReLU activations. While SPEAR is an important theoretical breakthrough, the attack's practicality was severely limited by its exponential runtime in the batch size b. In this work, we fill this gap by applying State-of-the-Art techniques from Sparsely-Used Dictionary Learning to make the problem of gradient inversion on linear layers with ReLU activations tractable. Our experiments demonstrate that our new attack, SPEAR++, retains all desirable properties of SPEAR, such as robustness to DP noise and FedAvg aggregation, while being applicable to 10x bigger batch sizes.

翻译：联邦学习近年来在现实场景中的应用日益广泛，因其能够在无需客户端间显式共享数据的情况下实现机器学习模型的分布式训练。然而，所谓的梯度反演攻击的出现从根本上挑战了其隐私保护特性。遗憾的是，由于这些攻击大多依赖于无形式化保证的直接数据优化，现实系统的脆弱性仍存争议，且每次新的联邦部署都需要进行繁琐的测试。为解决这些问题，近期提出的SPEAR攻击基于对带ReLU激活的线性层梯度的理论分析。尽管SPEAR是一项重要的理论突破，但其攻击实用性受限于随批次大小b呈指数级增长的计算时间。本研究通过应用稀疏使用字典学习领域的最新技术，使带ReLU激活的线性层梯度反演问题变得可解，从而填补了这一空白。实验表明，我们提出的新攻击方法SPEAR++保留了SPEAR的所有优良特性（如对差分隐私噪声和FedAvg聚合的鲁棒性），同时可应用于10倍更大的批次规模。