Recent advancements in federated learning (FL) seek to increase client-level performance by fine-tuning client parameters on local data or personalizing architectures for the local task. Existing methods for such personalization either prune a global model or fine-tune a global model on a local client distribution. However, these existing methods either personalize at the expense of retaining important global knowledge, or predetermine network layers for fine-tuning, resulting in suboptimal storage of global knowledge within client models. Enlightened by the lottery ticket hypothesis, we first introduce a hypothesis for finding optimal client subnetworks to locally fine-tune while leaving the rest of the parameters frozen. We then propose a novel FL framework, FedSelect, using this procedure that directly personalizes both client subnetwork structure and parameters, via the simultaneous discovery of optimal parameters for personalization and the rest of parameters for global aggregation during training. We show that this method achieves promising results on CIFAR-10.

翻译：联邦学习（FL）的最新进展旨在通过基于本地数据微调客户端参数或针对本地任务个性化架构来提升客户端级别的性能。现有的个性化方法通常对全局模型进行剪枝，或在本地客户端分布上微调全局模型。然而，这些现有方法要么以牺牲保留重要全局知识为代价实现个性化，要么预先确定用于微调的网络层，导致客户端模型中全局知识存储的次优性。受彩票假说启发，我们首先提出一种假设，用于寻找最优的客户端子网络进行本地微调，同时冻结其余参数。基于此流程，我们提出了一种新型联邦学习框架FedSelect，该框架通过训练过程中同时发现用于个性化的最优参数以及用于全局聚合的其余参数，直接对客户端子网络结构与参数进行个性化定制。实验表明，该方法在CIFAR-10数据集上取得了令人满意的性能。