Data heterogeneity across clients is one of the key challenges in Federated Learning (FL), which may slow down the global model convergence and even weaken global model performance. Most existing approaches tackle the heterogeneity by constraining local model updates through reference to global information provided by the server. This can alleviate the performance degradation on the aggregated global model. Different from existing methods, we focus the information exchange between clients, which could also enhance the effectiveness of local training and lead to generate a high-performance global model. Concretely, we propose a novel FL framework named FedCME by client matching and classifier exchanging. In FedCME, clients with large differences in data distribution will be matched in pairs, and then the corresponding pair of clients will exchange their classifiers at the stage of local training in an intermediate moment. Since the local data determines the local model training direction, our method can correct update direction of classifiers and effectively alleviate local update divergence. Besides, we propose feature alignment to enhance the training of the feature extractor. Experimental results demonstrate that FedCME performs better than FedAvg, FedProx, MOON and FedRS on popular federated learning benchmarks including FMNIST and CIFAR10, in the case where data are heterogeneous.

翻译：客户端间的数据异质性是联邦学习（Federated Learning, FL）的关键挑战之一，它可能拖慢全局模型收敛速度，甚至削弱全局模型性能。现有方法大多通过参考服务器提供的全局信息约束局部模型更新来应对异质性，这能缓解聚合后的全局模型性能退化。与现有方法不同，我们聚焦客户端之间的信息交换——这同样能增强局部训练效果，并有助于生成高性能全局模型。具体而言，我们提出了一种名为FedCME的新型联邦学习框架，通过客户端匹配与分类器交换实现。在FedCME中，数据分布差异较大的客户端将成对匹配，随后匹配的客户端对会在局部训练的中间阶段交换各自的分类器。由于局部数据决定了局部模型的训练方向，我们的方法能修正分类器的更新方向，有效缓解局部更新发散。此外，我们还提出了特征对齐方法以增强特征提取器的训练。实验结果表明，在数据异质的情况下，FedCME在FMNIST和CIFAR10等主流联邦学习基准测试中的表现优于FedAvg、FedProx、MOON和FedRS。