This paper jointly considers privacy preservation and Byzantine-robustness in decentralized learning. In a decentralized network, honest-but-curious agents faithfully follow the prescribed algorithm, but expect to infer their neighbors' private data from messages received during the learning process, while dishonest-and-Byzantine agents disobey the prescribed algorithm, and deliberately disseminate wrong messages to their neighbors so as to bias the learning process. For this novel setting, we investigate a generic privacy-preserving and Byzantine-robust decentralized stochastic gradient descent (SGD) framework, in which Gaussian noise is injected to preserve privacy and robust aggregation rules are adopted to counteract Byzantine attacks. We analyze its learning error and privacy guarantee, discovering an essential tradeoff between privacy preservation and Byzantine-robustness in decentralized learning -- the learning error caused by defending against Byzantine attacks is exacerbated by the Gaussian noise added to preserve privacy. Numerical experiments are conducted and corroborate our theoretical findings.

翻译：本文联合考虑去中心化学习中的隐私保护与拜占庭鲁棒性问题。在去中心化网络中，诚实但好奇的智能体忠实遵循预设算法，但试图从学习过程中接收的消息推断其邻居的私有数据；而不诚实且拜占庭的智能体则违反预设算法，故意向其邻居散布错误消息以偏置学习过程。针对这一新颖的场景，我们研究了一个通用的隐私保护且拜占庭鲁棒的去中心化随机梯度下降（SGD）框架，其中注入高斯噪声以保护隐私，并采用鲁棒聚合规则来对抗拜占庭攻击。我们分析了其学习误差与隐私保障，发现了去中心化学习中隐私保护与拜占庭鲁棒性之间的一种根本性权衡——由防御拜占庭攻击引起的学习误差会因添加的用于保护隐私的高斯噪声而加剧。我们进行了数值实验，实验结果印证了我们的理论发现。