Federated learning (FL) has become a popular means for distributed learning at clients using local data samples. However, recent studies have shown that FL may experience slow learning and poor performance when client data are not independent and identically distributed (IID). This paper proposes a new federated learning algorithm, where the central server has access to a small dataset, learns from it, and fuses the knowledge into the global model through the federated learning process. This new approach, referred to as Federated learning with Server Learning or FSL, is complementary to and can be combined with other FL learning algorithms. We prove the convergence of FSL and demonstrate its benefits through analysis and simulations. We also reveal an inherent trade-off: when the current model is far from any local minimizer, server learning can significantly improve and accelerate FL. On the other hand, when the model is close to a local minimizer, server learning could potentially affect the convergence neighborhood of FL due to variances in the estimated gradient used by the server. We show via simulations that such trade-off can be tuned easily to provide significant benefits, even when the server dataset is very small.

翻译：联邦学习(FL)已成为利用当地数据样本向客户提供分散学习的流行手段,然而,最近的研究表明,当客户数据不独立和同样分布(IID)时,FL可能会经历缓慢的学习和不良的绩效。本文提出了一个新的联合学习算法,中央服务器可以进入一个小数据集,从中学习,并通过联合学习过程将知识融入全球模型。这个称为“与服务器学习或FSL学习的Freed学习”的新方法,是与其他FL学习算法的补充,可以与其他FL学习算法相结合。我们证明FSL的趋同,并通过分析和模拟来证明它的好处。我们还揭示了一个内在的权衡:当目前的模型远离任何当地的最小化器时,服务器学习可以大大改进和加速FL。另一方面,当模型接近当地最小化器时,由于服务器使用的估计梯度的差异,服务器学习可能会影响FL的趋同区。我们通过模拟表明,这种交易可以很容易调整以提供显著的好处,即使服务器数据设置非常小。