Federated Learning (FL) is a distributed machine learning framework to alleviate the data silos, where decentralized clients collaboratively learn a global model without sharing their private data. However, the clients' Non-Independent and Identically Distributed (Non-IID) data negatively affect the trained model, and clients with different numbers of local updates may cause significant gaps to the local gradients in each communication round. In this paper, we propose a Federated Vectorized Averaging (FedVeca) method to address the above problem on Non-IID data. Specifically, we set a novel objective for the global model which is related to the local gradients. The local gradient is defined as a bi-directional vector with step size and direction, where the step size is the number of local updates and the direction is divided into positive and negative according to our definition. In FedVeca, the direction is influenced by the step size, thus we average the bi-directional vectors to reduce the effect of different step sizes. Then, we theoretically analyze the relationship between the step sizes and the global objective, and obtain upper bounds on the step sizes per communication round. Based on the upper bounds, we design an algorithm for the server and the client to adaptively adjusts the step sizes that make the objective close to the optimum. Finally, we conduct experiments on different datasets, models and scenarios by building a prototype system, and the experimental results demonstrate the effectiveness and efficiency of the FedVeca method.

翻译：联邦学习（FL）是一种分布式机器学习框架，旨在缓解数据孤岛问题，其中去中心化的客户端在无需共享私有数据的情况下协同学习全局模型。然而，客户端的非独立同分布（Non-IID）数据会对训练模型产生负面影响，且不同本地更新次数的客户端可能导致每轮通信中局部梯度存在显著差异。本文提出一种联邦向量化平均（FedVeca）方法来解决非IID数据下的上述问题。具体而言，我们为全局模型设定了一个与局部梯度相关的新目标函数。局部梯度被定义为具有步长和方向的双向向量，其中步长为本地更新次数，方向根据我们的定义分为正向和负向。在FedVeca中，方向受步长影响，因此我们对双向向量进行平均以减小不同步长的影响。随后，我们从理论上分析了步长与全局目标之间的关系，并得到了每轮通信步长的上界。基于该上界，我们设计了一种服务器与客户端自适应调整步长的算法，使目标函数接近最优值。最后，我们通过构建原型系统在不同数据集、模型和场景下开展实验，实验结果验证了FedVeca方法的有效性与高效性。