Federated learning is a popular framework for collaborative machine learning where multiple clients only share gradient updates on their local data with the server and not the actual data. Unfortunately, it was recently shown that gradient inversion attacks can reconstruct this data from these shared gradients. Existing attacks enable exact reconstruction only for a batch size of $b=1$ in the important honest-but-curious setting, with larger batches permitting only approximate reconstruction. In this work, we propose \emph{the first algorithm reconstructing whole batches with $b >1$ exactly}. This approach combines mathematical insights into the explicit low-rank structure of gradients with a sampling-based algorithm. Crucially, we leverage ReLU-induced gradient sparsity to precisely filter out large numbers of incorrect samples, making a final reconstruction step tractable. We provide an efficient GPU implementation for fully connected networks and show that it recovers batches of $b \lesssim 25$ elements exactly while being tractable for large network widths and depths.

翻译：联邦学习是一种流行的协作式机器学习框架，其中多个客户端仅向服务器共享其本地数据的梯度更新，而非实际数据。但近期研究表明，梯度反演攻击可从这些共享梯度中重构数据。现有关键诚实且好奇设定下，仅对批次大小$b=1$实现精确重构，更大批次则仅能近似重构。本文提出*首个能够精确重构$b>1$的完整批次的算法*。该方法融合了梯度显式低秩结构的数学洞察与基于采样的算法。关键创新在于利用ReLU激活函数诱导的梯度稀疏性，精准过滤大量错误样本，从而确保最终重构步骤的可行性。我们为全连接网络提供高效GPU实现，证明该方法可精确重构$b \lesssim 25$元素的批次，同时在大宽度与深度的网络中保持可计算性。