Federated learning is a distributed collaborative machine learning paradigm that has gained strong momentum in recent years. In federated learning, a central server periodically coordinates models with clients and aggregates the models trained locally by clients without necessitating access to local data. Despite its potential, the implementation of federated learning continues to encounter several challenges, predominantly the slow convergence that is largely due to data heterogeneity. The slow convergence becomes particularly problematic in cross-device federated learning scenarios where clients may be strongly limited by computing power and storage space, and hence counteracting methods that induce additional computation or memory cost on the client side such as auxiliary objective terms and larger training iterations can be impractical. In this paper, we propose a novel federated aggregation strategy, TurboSVM-FL, that poses no additional computation burden on the client side and can significantly accelerate convergence for federated classification task, especially when clients are "lazy" and train their models solely for few epochs for next global aggregation. TurboSVM-FL extensively utilizes support vector machine to conduct selective aggregation and max-margin spread-out regularization on class embeddings. We evaluate TurboSVM-FL on multiple datasets including FEMNIST, CelebA, and Shakespeare using user-independent validation with non-iid data distribution. Our results show that TurboSVM-FL can significantly outperform existing popular algorithms on convergence rate and reduce communication rounds while delivering better test metrics including accuracy, F1 score, and MCC.

翻译：联邦学习是一种分布式协作机器学习范式，近年来发展势头强劲。在联邦学习中，中央服务器定期与客户端协调模型，并聚合客户端本地训练的模型，无需访问本地数据。尽管潜力巨大，联邦学习的实施仍面临若干挑战，尤其是主要由数据异构性导致的收敛缓慢问题。这种缓慢收敛在跨设备联邦学习场景中尤为棘手——当客户端计算能力和存储空间严重受限时，引入额外计算或内存开销的对抗方法（如辅助目标项、增加训练迭代次数）可能不切实际。本文提出一种新型联邦聚合策略TurboSVM-FL，该策略不会给客户端增加额外计算负担，且能显著加速联邦分类任务的收敛，尤其是当客户端"懒惰"且仅训练少量轮次模型便参与下一轮全局聚合时。TurboSVM-FL充分利用支持向量机实现选择性聚合，并对类别嵌入进行最大间隔分散正则化。我们使用非独立同分布数据分布的独立用户验证方法，在FEMNIST、CelebA和Shakespeare等多个数据集上评估TurboSVM-FL。结果表明，TurboSVM-FL在收敛速度和通信轮次方面显著优于现有主流算法，同时能获得更好的测试指标（包括准确率、F1分数和MCC）。