Federated learning (FL) is a distributed machine learning paradigm that needs collaboration between a server and a series of clients with decentralized data. To make FL effective in real-world applications, existing work devotes to improving the modeling of decentralized data with non-independent and identical distributions (non-IID). In non-IID settings, there are intra-client inconsistency that comes from the imbalanced data modeling, and inter-client inconsistency among heterogeneous client distributions, which not only hinders sufficient representation of the minority data, but also brings discrepant model deviations. However, previous work overlooks to tackle the above two coupling inconsistencies together. In this work, we propose FedRANE, which consists of two main modules, i.e., local relational augmentation (LRA) and global Nash equilibrium (GNE), to resolve intra- and inter-client inconsistency simultaneously. Specifically, in each client, LRA mines the similarity relations among different data samples and enhances the minority sample representations with their neighbors using attentive message passing. In server, GNE reaches an agreement among inconsistent and discrepant model deviations from clients to server, which encourages the global model to update in the direction of global optimum without breaking down the clients optimization toward their local optimums. We conduct extensive experiments on four benchmark datasets to show the superiority of FedRANE in enhancing the performance of FL with non-IID data.

翻译：联邦学习（FL）是一种分布式机器学习范式，需要服务器与一系列拥有分散数据的客户端协同工作。为使FL在实际应用中有效，现有工作致力于改进对具有非独立同分布（non-IID）特征的分散数据的建模。在非IID场景中，存在由不平衡数据建模引起的客户端内部不一致性，以及由异质客户端分布导致的客户端间不一致性，这不仅阻碍了对少数数据的充分表示，还带来了有差异的模型偏差。然而，以往工作未考虑同时解决上述两种耦合的不一致性。本文提出FedRANE，该框架包含两个主要模块，即局部关系增强（LRA）和全局纳什均衡（GNE），以同时解决客户端内部与客户端间的不一致性。具体而言，在每个客户端中，LRA挖掘不同数据样本间的相似关系，并利用注意力消息传递增强少数样本的邻域表示。在服务器端，GNE在客户端向服务器传输的有差异且不一致的模型偏差间达成一致，促使全局模型沿全局最优方向更新，同时不破坏客户端向各自局部最优的优化进程。我们在四个基准数据集上进行了广泛实验，结果表明FedRANE在提升非IID数据下FL性能方面具有优越性。