Federated learning (FL) is a paradigm that allows several client devices and a server to collaboratively train a global model, by exchanging only model updates, without the devices sharing their local training data. These devices are often constrained in terms of communication and computation resources, and can further benefit from model pruning -- a paradigm that is widely used to reduce the size and complexity of models. Intuitively, by making local models coarser, pruning is expected to also provide some protection against privacy attacks in the context of FL. However this protection has not been previously characterized, formally or experimentally, and it is unclear if it is sufficient against state-of-the-art attacks. In this paper, we perform the first investigation of privacy guarantees for model pruning in FL. We derive information-theoretic upper bounds on the amount of information leaked by pruned FL models. We complement and validate these theoretical findings, with comprehensive experiments that involve state-of-the-art privacy attacks, on several state-of-the-art FL pruning schemes, using benchmark datasets. This evaluation provides valuable insights into the choices and parameters that can affect the privacy protection provided by pruning. Based on these insights, we introduce PriPrune -- a privacy-aware algorithm for local model pruning, which uses a personalized per-client defense mask and adapts the defense pruning rate so as to jointly optimize privacy and model performance. PriPrune is universal in that can be applied after any pruned FL scheme on the client, without modification, and protects against any inversion attack by the server. Our empirical evaluation demonstrates that PriPrune significantly improves the privacy-accuracy tradeoff compared to state-of-the-art pruned FL schemes that do not take privacy into account.

翻译：联邦学习（FL）是一种范式，允许多个客户端设备与服务器协同训练全局模型，仅交换模型更新而无需设备共享本地训练数据。这些设备通常受限于通信与计算资源，可进一步受益于模型剪枝——一种广泛用于降低模型规模与复杂度的范式。直观上，剪枝通过使本地模型更加粗糙，有望在FL中提供一定程度的隐私攻击防护。然而，这种防护此前未被从形式化或实验角度加以刻画，且尚不明确其是否足以抵御最先进的攻击。本文首次对FL中模型剪枝的隐私保障进行研究。我们从信息论角度推导了剪枝FL模型泄露信息量的上界。通过涉及最先进隐私攻击的综合实验，利用基准数据集对多种最先进的FL剪枝方案进行验证，补充并确认了这些理论发现。该评估为影响剪枝所提供隐私保护的选择和参数提供了宝贵见解。基于这些见解，我们提出PriPrune——一种面向本地模型剪枝的隐私感知算法，该算法采用个性化客户端防御掩码并自适应调整防御剪枝率，以联合优化隐私与模型性能。PriPrune具有通用性，可无需修改地应用于客户端上任何剪枝FL方案之后，并抵御服务器的任意反演攻击。实验评估表明，与未考虑隐私的最先进剪枝FL方案相比，PriPrune显著改善了隐私-准确率权衡。