This paper jointly considers privacy preservation and Byzantine-robustness in decentralized learning. In a decentralized network, honest-but-curious agents faithfully follow the prescribed algorithm, but expect to infer their neighbors' private data from messages received during the learning process, while dishonest-and-Byzantine agents disobey the prescribed algorithm, and deliberately disseminate wrong messages to their neighbors so as to bias the learning process. For this novel setting, we investigate a generic privacy-preserving and Byzantine-robust decentralized stochastic gradient descent (SGD) framework, in which Gaussian noise is injected to preserve privacy and robust aggregation rules are adopted to counteract Byzantine attacks. We analyze its learning error and privacy guarantee, discovering an essential tradeoff between privacy preservation and Byzantine-robustness in decentralized learning -- the learning error caused by defending against Byzantine attacks is exacerbated by the Gaussian noise added to preserve privacy. For a class of state-of-the-art robust aggregation rules, we give unified analysis of the "mixing abilities". Building upon this analysis, we reveal how the "mixing abilities" affect the tradeoff between privacy preservation and Byzantine-robustness. The theoretical results provide guidelines for achieving a favorable tradeoff with proper design of robust aggregation rules. Numerical experiments are conducted and corroborate our theoretical findings.

翻译：本文联合考虑了去中心化学习中的隐私保护与拜占庭鲁棒性问题。在去中心化网络中，诚实但好奇的智能体忠实地遵循预设算法，但期望从学习过程中接收的消息推断其邻居的私有数据；而不诚实且拜占庭式的智能体则不遵守预设算法，故意向其邻居传播错误消息以偏斜学习过程。针对这一新型场景，我们研究了一个通用的隐私保护且拜占庭鲁棒的分布式随机梯度下降（SGD）框架，其中注入高斯噪声以保护隐私，并采用鲁棒聚合规则来抵御拜占庭攻击。我们分析了其学习误差与隐私保证，发现了去中心化学习中隐私保护与拜占庭鲁棒性之间的本质权衡——由防御拜占庭攻击导致的学习误差会因添加用于保护隐私的高斯噪声而加剧。针对一类最先进的鲁棒聚合规则，我们给出了“混合能力”的统一分析。基于此分析，我们揭示了“混合能力”如何影响隐私保护与拜占庭鲁棒性之间的权衡。理论结果为通过合理设计鲁棒聚合规则实现有利的权衡提供了指导。数值实验得以进行，并验证了我们的理论发现。