As a promising learning paradigm integrating computation and communication, federated learning (FL) proceeds the local training and the periodic sharing from distributed clients. Due to the non-i.i.d. data distribution on clients, FL model suffers from the gradient diversity, poor performance, bad convergence, etc. In this work, we aim to tackle this key issue by adopting importance sampling (IS) for local training. We propose importance sampling federated learning (ISFL), an explicit framework with theoretical guarantees. Firstly, we derive the convergence theorem of ISFL to involve the effects of local importance sampling. Then, we formulate the problem of selecting optimal IS weights and obtain the theoretical solutions. We also employ a water-filling method to calculate the IS weights and develop the ISFL algorithms. The experimental results on CIFAR-10 fit the proposed theorems well and verify that ISFL reaps better performance, sampling efficiency, as well as explainability on non-i.i.d. data. To the best of our knowledge, ISFL is the first non-i.i.d. FL solution from the local sampling aspect which exhibits theoretical compatibility with neural network models. Furthermore, as a local sampling approach, ISFL can be easily migrated into other emerging FL frameworks.

翻译：联邦学习作为一种整合计算与通信的具有前景的学习范式，通过分布式客户端的本地训练与周期性共享来推进。由于客户端上的非独立同分布数据分布，联邦学习模型面临梯度多样性、性能欠佳、收敛性差等问题。在本工作中，我们旨在通过引入重要性抽样进行局部训练来解决这一关键问题。我们提出了具有理论保证的明确框架——重要性抽样联邦学习（ISFL）。首先，我们推导了ISFL的收敛定理以纳入局部重要性抽样的影响。然后，我们公式化了最优IS权重的选择问题并获得了理论解。我们还采用注水法计算IS权重并开发了ISFL算法。在CIFAR-10上的实验结果与所提出的定理高度吻合，并验证了ISFL在非独立同分布数据上具有更优的性能、采样效率及可解释性。据我们所知，ISFL是首个从局部采样角度出发、在理论上兼容神经网络模型的非独立同分布联邦学习解决方案。此外，作为局部采样方法，ISFL可便捷地迁移至其他新兴联邦学习框架中。