Federated learning provides a privacy-aware learning framework by enabling participants to jointly train models without exposing their private data. However, federated learning has exhibited vulnerabilities to Byzantine attacks, where the adversary aims to destroy the convergence and performance of the global model. Meanwhile, we observe that most existing robust AGgregation Rules (AGRs) fail to stop the aggregated gradient deviating from the optimal gradient (the average of honest gradients) in the non-IID setting. We attribute the reason of the failure of these AGRs to two newly proposed concepts: identification failure and integrity failure. The identification failure mainly comes from the exacerbated curse of dimensionality in the non-IID setting. The integrity failure is a combined result of conservative filtering strategy and gradient heterogeneity. In order to address both failures, we propose GAIN, a gradient decomposition scheme that can help adapt existing robust algorithms to heterogeneous datasets. We also provide convergence analysis for integrating existing robust AGRs into GAIN. Experiments on various real-world datasets verify the efficacy of our proposed GAIN.

翻译：联邦学习通过允许参与方在不暴露私有数据的情况下联合训练模型，提供了一种隐私感知的学习框架。然而，联邦学习表现出易受拜占庭攻击的脆弱性，攻击者旨在破坏全局模型的收敛性与性能。同时，我们观察到在非独立同分布（non-IID）场景下，大多数现有鲁棒聚合规则（AGRs）无法阻止聚合梯度偏离最优梯度（即诚实梯度的平均值）。我们将这些AGR失效的原因归结为两个新提出的概念：识别失效与完整性失效。识别失效主要源于非IID场景下加剧的维度诅咒，而完整性失效则是保守过滤策略与梯度异质性的综合结果。为解决上述两种失效，我们提出GAIN——一种梯度分解方案，可帮助现有鲁棒算法适应异质数据集。我们还提供了将现有鲁棒AGR集成至GAIN的收敛性分析。在多种真实世界数据集上的实验验证了所提GAIN方法的有效性。