Learning a global model by abstracting the knowledge, distributed across multiple clients, without aggregating the raw data is the primary goal of Federated Learning (FL). Typically, this works in rounds alternating between parallel local training at several clients, followed by model aggregation at a server. We found that existing FL methods under-perform when local datasets are small and present severe label skew as these lead to over-fitting and local model bias. This is a realistic setting in many real-world applications. To address the problem, we propose \textit{FLea}, a unified framework that tackles over-fitting and local bias by encouraging clients to exchange privacy-protected features to aid local training. The features refer to activations from an intermediate layer of the model, which are obfuscated before being shared with other clients to protect sensitive information in the data. \textit{FLea} leverages a novel way of combining local and shared features as augmentations to enhance local model learning. Our extensive experiments demonstrate that \textit{FLea} outperforms the start-of-the-art FL methods, sharing only model parameters, by up to $17.6\%$, and FL methods that share data augmentations by up to $6.3\%$, while reducing the privacy vulnerability associated with shared data augmentations.

翻译：联邦学习（FL）的核心目标是在不聚合原始数据的前提下，通过抽象分布在各客户端上的知识来学习全局模型。通常，该过程以轮次交替进行：先在多个客户端并行执行本地训练，随后在服务器端进行模型聚合。我们发现，当本地数据集规模较小且存在严重标签偏斜时，现有联邦学习方法表现不佳——这两种因素会导致过拟合和本地模型偏差。这恰是许多实际应用中的真实场景。为解决该问题，我们提出统一框架\textit{FLea}，通过鼓励客户端交换受隐私保护的特征来辅助本地训练，从而应对过拟合和局部偏差。这里的特征指模型中间层的激活值，在共享前经过混淆处理以保护数据中的敏感信息。\textit{FLea}采用新颖方式将本地特征与共享特征结合作为数据增强手段，以提升本地模型学习效果。广泛实验表明，\textit{FLea}相比仅共享模型参数的最先进联邦学习方法性能提升高达$17.6\%$，相比共享数据增强的联邦学习方法提升达$6.3\%$，同时降低了共享数据增强带来的隐私风险。