One of the challenges in federated learning is the non-independent and identically distributed (non-iid) characteristics between heterogeneous devices, which cause significant differences in local updates and affect the performance of the central server. Although many studies have been proposed to address this challenge, they only focus on local training and aggregation processes to smooth the changes and fail to achieve high performance with deep learning models. Inspired by the phenomenon of neural collapse, we force each client to be optimized toward an optimal global structure for classification. Specifically, we initialize it as a random simplex Equiangular Tight Frame (ETF) and fix it as the unit optimization target of all clients during the local updating. After guaranteeing all clients are learning to converge to the global optimum, we propose to add a global memory vector for each category to remedy the parameter fluctuation caused by the bias of the intra-class condition distribution among clients. Our experimental results show that our method can improve the performance with faster convergence speed on different-size datasets.

翻译：联邦学习面临的一个挑战是异构设备间的非独立同分布 (non-iid) 特性，该特性会导致局部更新产生显著差异，从而影响中央服务器的性能。尽管已有诸多研究试图应对这一挑战，但现有方法仅聚焦于局部训练和聚合过程以平滑变化，未能借助深度学习模型实现高性能。受神经坍缩现象的启发，我们强制每个客户端向最优全局分类结构优化。具体而言，我们将其初始化为随机单纯形等角紧框架 (Equiangular Tight Frame, ETF)，并将其固定为所有客户端在局部更新过程中的统一优化目标。在确保所有客户端学习收敛至全局最优后，我们进一步提出为每个类别添加全局记忆向量，以修复由客户端间类内条件分布偏差引起的参数波动。实验结果表明，我们的方法能够提升不同规模数据集上的性能，并实现更快的收敛速度。