Federated Learning (FL) aims to train machine learning models for multiple clients without sharing their own private data. Due to the heterogeneity of clients' local data distribution, recent studies explore the personalized FL that learns and deploys distinct local models with the help of auxiliary global models. However, the clients can be heterogeneous in terms of not only local data distribution, but also their computation and communication resources. The capacity and efficiency of personalized models are restricted by the lowest-resource clients, leading to sub-optimal performance and limited practicality of personalized FL. To overcome these challenges, we propose a novel approach named pFedGate for efficient personalized FL by adaptively and efficiently learning sparse local models. With a lightweight trainable gating layer, pFedGate enables clients to reach their full potential in model capacity by generating different sparse models accounting for both the heterogeneous data distributions and resource constraints. Meanwhile, the computation and communication efficiency are both improved thanks to the adaptability between the model sparsity and clients' resources. Further, we theoretically show that the proposed pFedGate has superior complexity with guaranteed convergence and generalization error. Extensive experiments show that pFedGate achieves superior global accuracy, individual accuracy and efficiency simultaneously over state-of-the-art methods. We also demonstrate that pFedGate performs better than competitors in the novel clients participation and partial clients participation scenarios, and can learn meaningful sparse local models adapted to different data distributions.

翻译：联邦学习（FL）旨在在多个客户端间训练机器学习模型，同时无需共享各方的私有数据。鉴于客户端本地数据分布的异质性，近年来研究探索了个性化联邦学习，通过借助辅助全局模型来学习并部署不同的本地模型。然而，客户端不仅在本地数据分布上存在异质性，其计算与通信资源亦各不相同。个性化模型的容量与效率受限于资源最薄弱的客户端，导致个性化联邦学习的性能次优且实用性受限。为克服上述挑战，我们提出一种名为pFedGate的新型高效个性化联邦学习方法，该方法通过自适应且高效地学习稀疏本地模型来实现目标。借助轻量级可训练门控层，pFedGate可使客户端通过生成兼顾异构数据分布与资源约束的不同稀疏模型，充分释放模型容量潜能。同时，由于模型稀疏性与客户端资源间的自适应匹配，计算效率与通信效率均得到提升。进一步，我们从理论上证明，所提出的pFedGate在保证收敛性与泛化误差的前提下具有优越的复杂度。大量实验表明，pFedGate在全局准确率、个体准确率及效率方面均优于现有最优方法。我们亦验证了pFedGate在新型客户端参与及部分客户端参与场景中表现优于其他方法，并能学习到适应不同数据分布的有意义稀疏本地模型。