Federated learning (FL) has recently become a hot research topic, in which Byzantine robustness, communication efficiency and privacy preservation are three important aspects. However, the tension among these three aspects makes it hard to simultaneously take all of them into account. In view of this challenge, we theoretically analyze the conditions that a communication compression method should satisfy to be compatible with existing Byzantine-robust methods and privacy-preserving methods. Motivated by the analysis results, we propose a novel communication compression method called consensus sparsification (ConSpar). To the best of our knowledge, ConSpar is the first communication compression method that is designed to be compatible with both Byzantine-robust methods and privacy-preserving methods. Based on ConSpar, we further propose a novel FL framework called FedREP, which is Byzantine-robust, communication-efficient and privacy-preserving. We theoretically prove the Byzantine robustness and the convergence of FedREP. Empirical results show that FedREP can significantly outperform communication-efficient privacy-preserving baselines. Furthermore, compared with Byzantine-robust communication-efficient baselines, FedREP can achieve comparable accuracy with the extra advantage of privacy preservation.

翻译：联邦学习（FL）近年来成为热门研究课题，其中拜占庭鲁棒性、通信效率与隐私保护是三个重要方面。然而，这三者之间的相互制约使得同时兼顾它们变得困难。针对这一挑战，我们从理论上分析了通信压缩方法需满足的条件，以使其能与现有拜占庭鲁棒方法和隐私保护方法兼容。基于分析结果，我们提出了一种名为共识稀疏化（ConSpar）的新型通信压缩方法。据我们所知，ConSpar是首个被设计为与拜占庭鲁棒方法和隐私保护方法均可兼容的通信压缩方法。基于ConSpar，我们进一步提出了一个名为FedREP的新型联邦学习框架，该框架兼具拜占庭鲁棒性、通信高效性和隐私保护能力。我们从理论上证明了FedREP的拜占庭鲁棒性与收敛性。实验结果表明，FedREP显著优于通信高效的隐私保护基线方法。此外，与拜占庭鲁棒的通信高效基线方法相比，FedREP在达到相近精度的同时，还额外具有隐私保护的优势。