Edge computing allows artificial intelligence and machine learning models to be deployed on edge devices, where they can learn from local data and collaborate to form a global model. Federated learning (FL) is a distributed machine learning technique that facilitates this process while preserving data privacy. However, FL also faces challenges such as high computational and communication costs regarding resource-constrained devices, and poor generalization performance due to the heterogeneity of data across edge clients and the presence of out-of-distribution data. In this paper, we propose the Gradient-Congruity Guided Federated Sparse Training (FedSGC), a novel method that integrates dynamic sparse training and gradient congruity inspection into federated learning framework to address these issues. Our method leverages the idea that the neurons, in which the associated gradients with conflicting directions with respect to the global model contain irrelevant or less generalized information for other clients, and could be pruned during the sparse training process. Conversely, the neurons where the associated gradients with consistent directions could be grown in a higher priority. In this way, FedSGC can greatly reduce the local computation and communication overheads while, at the same time, enhancing the generalization abilities of FL. We evaluate our method on challenging non-i.i.d settings and show that it achieves competitive accuracy with state-of-the-art FL methods across various scenarios while minimizing computation and communication costs.

翻译：边缘计算使得人工智能和机器学习模型能够部署在边缘设备上，这些设备可从本地数据中学习并协作形成全局模型。联邦学习（FL）作为一种分布式机器学习技术，在保障数据隐私的同时促进了这一过程。然而，FL也面临诸多挑战：资源受限设备的高计算与通信成本，以及因边缘客户端数据异构性和分布外数据导致的泛化性能不足。本文提出梯度协同引导的联邦稀疏训练（FedSGC），该方法将动态稀疏训练与梯度协同性检测集成至联邦学习框架中，旨在解决上述问题。我们的方法基于以下理念：若神经元关联梯度相对于全局模型存在方向冲突，则其包含对其他客户端无关或泛化性较差的信息，可在稀疏训练过程中被剪枝；反之，关联梯度方向一致的神经元应优先增长。通过这种方式，FedSGC能够显著降低本地计算与通信开销，同时增强FL的泛化能力。我们在具有挑战性的非独立同分布场景下评估了该方法，结果表明其在各类场景中能以最小化计算与通信成本为代价，达到与最先进FL方法相媲美的准确率。