Federated Learning (FL) algorithms using Knowledge Distillation (KD) have received increasing attention due to their favorable properties with respect to privacy, non-i.i.d. data and communication cost. These methods depart from transmitting model parameters and, instead, communicate information about a learning task by sharing predictions on a public dataset. In this work, we study the performance of such approaches in the byzantine setting, where a subset of the clients act in an adversarial manner aiming to disrupt the learning process. We show that KD-based FL algorithms are remarkably resilient and analyze how byzantine clients can influence the learning process compared to Federated Averaging. Based on these insights, we introduce two new byzantine attacks and demonstrate that they are effective against prior byzantine-resilient methods. Additionally, we propose FilterExp, a novel method designed to enhance the byzantine resilience of KD-based FL algorithms and demonstrate its efficacy. Finally, we provide a general method to make attacks harder to detect, improving their effectiveness.

翻译：联邦学习（FL）算法结合知识蒸馏（KD）因在隐私保护、非独立同分布数据及通信成本方面的优越特性而日益受到关注。此类方法不再传输模型参数，而是通过共享公共数据集上的预测结果来传递学习任务信息。本研究在拜占庭设置下考察了这些方法的性能——其中部分客户端以恶意方式行动，试图破坏学习过程。我们证明基于KD的联邦学习算法具有显著的容错性，并分析了相较于联邦平均算法，拜占庭客户端如何影响学习过程。基于这些发现，我们提出了两种新型拜占庭攻击，并证明其对现有拜占庭容错方法具有攻击效力。此外，我们设计了FilterExp——一种增强基于KD的联邦学习算法拜占庭容错性的新方法，并验证了其有效性。最后，我们提出了一种提升攻击隐蔽性的通用方法，有效增强了攻击效果。