Federated Learning (FL) enables multiple clients to collaboratively learn a machine learning model without exchanging their own local data. In this way, the server can exploit the computational power of all clients and train the model on a larger set of data samples among all clients. Although such a mechanism is proven to be effective in various fields, existing works generally assume that each client preserves sufficient data for training. In practice, however, certain clients may only contain a limited number of samples (i.e., few-shot samples). For example, the available photo data taken by a specific user with a new mobile device is relatively rare. In this scenario, existing FL efforts typically encounter a significant performance drop on these clients. Therefore, it is urgent to develop a few-shot model that can generalize to clients with limited data under the FL scenario. In this paper, we refer to this novel problem as federated few-shot learning. Nevertheless, the problem remains challenging due to two major reasons: the global data variance among clients (i.e., the difference in data distributions among clients) and the local data insufficiency in each client (i.e., the lack of adequate local data for training). To overcome these two challenges, we propose a novel federated few-shot learning framework with two separately updated models and dedicated training strategies to reduce the adverse impact of global data variance and local data insufficiency. Extensive experiments on four prevalent datasets that cover news articles and images validate the effectiveness of our framework compared with the state-of-the-art baselines. Our code is provided at https://github.com/SongW-SW/F2L.

翻译：联邦学习（Federated Learning, FL）使多个客户端能够在不交换本地数据的情况下协作学习机器学习模型。通过这种方式，服务器可以利用所有客户端的计算能力，并在更大规模的数据样本集上训练模型。尽管这种机制已被证明在各个领域行之有效，但现有工作通常假设每个客户端都拥有充足的数据用于训练。然而在实践中，某些客户端可能仅包含有限数量的样本（即小样本）。例如，特定用户使用新移动设备拍摄的照片数据相对稀少。在此场景下，现有联邦学习方法往往在这些客户端上出现显著的性能下降。因此，亟需开发一种能够在联邦学习场景下泛化到数据有限客户端的小样本模型。本文将此新问题称为联邦小样本学习。然而，该问题仍面临两大挑战：客户端间的全局数据差异（即各客户端数据分布差异）和每个客户端的本地数据不足（即缺乏足够的本地训练数据）。为克服这两个挑战，我们提出了一种新颖的联邦小样本学习框架，该框架包含两个独立更新的模型和专用训练策略，以降低全局数据差异和本地数据不足带来的不利影响。在涵盖新闻文章和图像的四个主流数据集上进行的广泛实验验证了我们的框架相比现有最优基线方法的有效性。我们的代码已开源在 https://github.com/SongW-SW/F2L。