As a promising learning paradigm integrating computation and communication, federated learning (FL) proceeds the local training and the periodic sharing from distributed clients. Due to the non-i.i.d. data distribution on clients, FL model suffers from the gradient diversity, poor performance, bad convergence, etc. In this work, we aim to tackle this key issue by adopting importance sampling (IS) for local training. We propose importance sampling federated learning (ISFL), an explicit framework with theoretical guarantees. Firstly, we derive the convergence theorem of ISFL to involve the effects of local importance sampling. Then, we formulate the problem of selecting optimal IS weights and obtain the theoretical solutions. We also employ a water-filling method to calculate the IS weights and develop the ISFL algorithms. The experimental results on CIFAR-10 fit the proposed theorems well and verify that ISFL reaps better performance, sampling efficiency, as well as explainability on non-i.i.d. data. To the best of our knowledge, ISFL is the first non-i.i.d. FL solution from the local sampling aspect which exhibits theoretical compatibility with neural network models. Furthermore, as a local sampling approach, ISFL can be easily migrated into other emerging FL frameworks.

翻译：作为一种融合计算与通信的富有前景的学习范式，联邦学习通过分布式客户端执行本地训练并周期性地共享模型参数。由于客户端间的数据分布呈现非独立同分布特性，联邦学习模型面临梯度多样性不足、性能下降、收敛性差等问题。本文通过将重要性采样引入本地训练过程来解决这一关键问题，提出具有理论保障的重要性采样联邦学习（ISFL）框架。首先，我们推导了ISFL的收敛定理，揭示了局部重要性采样对模型收敛的影响机理。其次，我们构建了最优重要性采样权重的选择问题并给出理论解，进而采用注水法计算重要性采样权重并开发ISFL算法。在CIFAR-10数据集上的实验结果与所提定理高度吻合，验证了ISFL在非独立同分布数据上具有更优性能、采样效率及可解释性。据我们所知，ISFL是首个从局部采样角度解决非独立同分布问题，且与神经网络模型保持理论兼容性的联邦学习方案。此外，作为局部采样方法，ISFL可便捷地迁移至其他新兴联邦学习框架中。