Federated Learning (FL) is an emerging paradigm that allows a model to be trained across a number of participants without sharing data. Recent works have begun to consider the effects of using pre-trained models as an initialization point for existing FL algorithms; however, these approaches ignore the vast body of efficient transfer learning literature from the centralized learning setting. Here we revisit the problem of FL from a pre-trained model considered in prior work and expand it to a set of computer vision transfer learning problems. We first observe that simply fitting a linear classification head can be efficient and effective in many cases. We then show that in the FL setting, fitting a classifier using the Nearest Class Means (NCM) can be done exactly and orders of magnitude more efficiently than existing proposals, while obtaining strong performance. Finally, we demonstrate that using a two-phase approach of obtaining the classifier and then fine-tuning the model can yield rapid convergence and improved generalization in the federated setting. We demonstrate the potential our method has to reduce communication and compute costs while achieving better model performance.

翻译：联邦学习作为一种新兴范式，允许在多个参与者间训练模型而无需共享数据。近期研究开始探讨将预训练模型作为初始化起点应用于现有联邦学习算法的效果，但这些方法忽视了集中式学习场景中大量高效的迁移学习文献。本文重新审视了前人工作中涉及的基于预训练模型的联邦学习问题，并将其拓展至计算机视觉迁移学习问题集。我们首先发现，在许多情况下，简单拟合线性分类头即可实现高效且有效的性能。进而证明，在联邦学习场景中，采用最近类均值法拟合分类器可精确实现，且计算效率比现有方案高出数量级，同时保持强劲性能。最后，我们证明采用先获取分类器再微调模型的两阶段方法，可在联邦学习场景中实现快速收敛和泛化能力提升。我们的实验表明，该方法在获得更优模型性能的同时，具有降低通信与计算成本的潜力。