Privacy-preserving federated learning allows multiple users to jointly train a model with coordination of a central server. The server only learns the final aggregation result, thus the users' (private) training data is not leaked from the individual model updates. However, keeping the individual updates private allows malicious users to perform Byzantine attacks and degrade the accuracy without being detected. Best existing defenses against Byzantine workers rely on robust rank-based statistics, e.g., median, to find malicious updates. However, implementing privacy-preserving rank-based statistics is nontrivial and not scalable in the secure domain, as it requires sorting all individual updates. We establish the first private robustness check that uses high break point rank-based statistics on aggregated model updates. By exploiting randomized clustering, we significantly improve the scalability of our defense without compromising privacy. We leverage our statistical bounds in zero-knowledge proofs to detect and remove malicious updates without revealing the private user updates. Our novel framework, zPROBE, enables Byzantine resilient and secure federated learning. Empirical evaluations demonstrate that zPROBE provides a low overhead solution to defend against state-of-the-art Byzantine attacks while preserving privacy.

翻译：隐私保护联邦学习允许多个用户通过中央服务器协调共同训练模型。服务器仅学习最终的聚合结果，因此用户的（私有）训练数据不会从单个模型更新中泄露。然而，对单个更新进行保密使得恶意用户能够实施拜占庭攻击并在不被察觉的情况下降低模型精度。现有针对拜占庭工作者的最佳防御依赖基于排序的鲁棒统计量（例如中位数）来识别恶意更新。然而，在安全域中实现隐私保护的基于排序的统计量既非平凡也难以扩展，因为它需要对所有单个更新进行排序。我们建立了首个在聚合模型更新上使用高破坏点排序统计量的隐私鲁棒性检查。通过利用随机聚类，我们显著提高了防御的可扩展性，同时不损害隐私。我们利用零知识证明中的统计界限来检测并移除恶意更新，同时不泄露私有用户更新。我们的新框架zPROBE实现了拜占庭容错且安全的联邦学习。实验评估表明，zPROBE在保护隐私的同时，能以低开销防御最先进的拜占庭攻击。