Federated learning (FL) is an appealing approach to training machine learning models without sharing raw data. However, standard FL algorithms are iterative and thus induce a significant communication cost. One-shot federated learning (OFL) trades the iterative exchange of models between clients and the server with a single round of communication, thereby saving substantially on communication costs. Not surprisingly, OFL exhibits a performance gap in terms of accuracy with respect to FL, especially under high data heterogeneity. We introduce FENS, a novel federated ensembling scheme that approaches the accuracy of FL with the communication efficiency of OFL. Learning in FENS proceeds in two phases: first, clients train models locally and send them to the server, similar to OFL; second, clients collaboratively train a lightweight prediction aggregator model using FL. We showcase the effectiveness of FENS through exhaustive experiments spanning several datasets and heterogeneity levels. In the particular case of heterogeneously distributed CIFAR-10 dataset, FENS achieves up to a 26.9% higher accuracy over state-of-the-art (SOTA) OFL, being only 3.1% lower than FL. At the same time, FENS incurs at most 4.3x more communication than OFL, whereas FL is at least 10.9x more communication-intensive than FENS.

翻译：联邦学习（FL）是一种无需共享原始数据即可训练机器学习模型的吸引人方法。然而，标准联邦学习算法是迭代式的，因此会产生显著的通信开销。单次联邦学习（OFL）通过单轮通信替代了客户端与服务器之间模型的迭代交换，从而大幅节省了通信成本。不出所料，OFL在准确率方面与FL存在性能差距，尤其是在数据高度异构的情况下。我们提出了FENS，一种新颖的联邦集成方案，它能够以OFL的通信效率逼近FL的准确率。FENS的学习过程分为两个阶段：首先，客户端在本地训练模型并将其发送至服务器，类似于OFL；其次，客户端使用联邦学习协作训练一个轻量级的预测聚合器模型。我们通过在多个数据集和异构水平上的详尽实验展示了FENS的有效性。在CIFAR-10数据集异构分布的具体案例中，FENS相较于最先进的OFL方法实现了高达26.9%的准确率提升，仅比FL低3.1%。同时，FENS产生的通信开销最多仅为OFL的4.3倍，而FL的通信开销至少是FENS的10.9倍。