Federated Learning (FL) has become an established technique to facilitate privacy-preserving collaborative training. However, new approaches to FL often discuss their contributions involving small deep-learning models only. With the tremendous success of transformer models, the following question arises: What is necessary to operationalize foundation models in an FL application? Knowing that computation and communication often take up similar amounts of time in FL, we introduce a novel taxonomy focused on computational and communication efficiency methods in FL applications. This said, these methods aim to optimize the training time and reduce communication between clients and the server. We also look at the current state of widely used FL frameworks and discuss future research potentials based on existing approaches in FL research and beyond.

翻译：联邦学习（FL）已成为一种促进隐私保护协作训练的成熟技术。然而，新的联邦学习方法通常仅针对小型深度学习模型讨论其贡献。随着Transformer模型的巨大成功，以下问题随之而来：在联邦学习应用中，如何实现基础模型的实际应用？考虑到在联邦学习中计算和通信往往消耗相近的时间，我们提出了一种新颖的分类法，重点关注联邦学习应用中的计算与通信效率方法。这些方法旨在优化训练时间并减少客户端与服务器之间的通信。我们还审视了当前广泛使用的联邦学习框架的现状，并基于现有联邦学习研究及其相关领域的方法，探讨了未来的研究潜力。