Collaborative learning (CL) is a distributed learning framework that aims to protect user privacy by allowing users to jointly train a model by sharing their gradient updates only. However, gradient inversion attacks (GIAs), which recover users' training data from shared gradients, impose severe privacy threats to CL. Existing defense methods adopt different techniques, e.g., differential privacy, cryptography, and perturbation defenses, to defend against the GIAs. Nevertheless, all current defense methods suffer from a poor trade-off between privacy, utility, and efficiency. To mitigate the weaknesses of existing solutions, we propose a novel defense method, Dual Gradient Pruning (DGP), based on gradient pruning, which can improve communication efficiency while preserving the utility and privacy of CL. Specifically, DGP slightly changes gradient pruning with a stronger privacy guarantee. And DGP can also significantly improve communication efficiency with a theoretical analysis of its convergence and generalization. Our extensive experiments show that DGP can effectively defend against the most powerful GIAs and reduce the communication cost without sacrificing the model's utility.

翻译：协作学习（CL）是一种分布式学习框架，旨在通过仅允许用户共享梯度更新来联合训练模型，从而保护用户隐私。然而，梯度反转攻击（GIA）能从共享梯度中恢复用户的训练数据，给CL带来严重的隐私威胁。现有防御方法采用不同技术（例如差分隐私、密码学和扰动防御）来抵御GIA。尽管如此，当前所有防御方法在隐私、效用和效率之间均存在较差的权衡。为缓解现有方案的不足，我们基于梯度剪枝提出了一种新颖的防御方法——双重梯度剪枝（DGP），该方法能在保持CL的效用和隐私的同时提升通信效率。具体来说，DGP对梯度剪枝进行微调，使其具有更强的隐私保障。同时，DGP通过收敛性与泛化性的理论分析，能显著提升通信效率。大量实验表明，DGP可有效抵御最强大的GIA，并在不牺牲模型效用的前提下降低通信成本。