Recent advancements in federated learning (FL) seek to increase client-level performance by fine-tuning client parameters on local data or personalizing architectures for the local task. Existing methods for such personalization either prune a global model or fine-tune a global model on a local client distribution. However, these existing methods either personalize at the expense of retaining important global knowledge, or predetermine network layers for fine-tuning, resulting in suboptimal storage of global knowledge within client models. Enlightened by the lottery ticket hypothesis, we first introduce a hypothesis for finding optimal client subnetworks to locally fine-tune while leaving the rest of the parameters frozen. We then propose a novel FL framework, FedSelect, using this procedure that directly personalizes both client subnetwork structure and parameters, via the simultaneous discovery of optimal parameters for personalization and the rest of parameters for global aggregation during training. We show that this method achieves promising results on CIFAR-10.

翻译：近期联邦学习（FL）的进展旨在通过客户端本地数据微调参数或针对本地任务个性化架构，来提升客户端级别的性能。现有的个性化方法要么对全局模型进行剪枝，要么在本地客户端分布上微调全局模型。然而，这些现有方法要么在保留重要全局知识方面牺牲了个性化效果，要么预设网络层进行微调，导致全局知识在客户端模型中的存储次优。受彩票假说的启发，我们首先提出一个假设，用于寻找最优的客户端子网络进行本地微调，同时保持其余参数冻结。基于此，我们提出一种新型联邦学习框架FedSelect，该框架通过训练过程中同时发现用于个性化的最优参数以及用于全局聚合的其余参数，直接对客户端子网络结构和参数进行个性化。实验表明，该方法在CIFAR-10数据集上取得了显著成效。