We propose a novel contrastive learning framework to effectively address the challenges of data heterogeneity in federated learning. We first analyze the inconsistency of gradient updates across clients during local training and establish its dependence on the distribution of feature representations, leading to the derivation of the supervised contrastive learning (SCL) objective to mitigate local deviations. In addition, we show that a na\"ive adoption of SCL in federated learning leads to representation collapse, resulting in slow convergence and limited performance gains. To address this issue, we introduce a relaxed contrastive learning loss that imposes a divergence penalty on excessively similar sample pairs within each class. This strategy prevents collapsed representations and enhances feature transferability, facilitating collaborative training and leading to significant performance improvements. Our framework outperforms all existing federated learning approaches by huge margins on the standard benchmarks through extensive experimental results.

翻译：我们提出了一种新颖的对比学习框架，以有效解决联邦学习中数据异构性的挑战。首先，我们分析了本地训练过程中各客户端梯度更新的不一致性，并确定其与特征表示分布的依赖关系，从而推导出监督对比学习目标以缓解局部偏离。此外，我们发现在联邦学习中直接采用监督对比学习会导致表示坍塌，进而引发收敛缓慢和性能提升有限的问题。为解决该问题，我们引入了一种放松对比学习损失，该损失对每个类别内过于相似的样本对施加发散惩罚。此策略可防止表示坍塌并增强特征迁移性，从而促进协作训练并带来显著的性能提升。通过大量实验，我们的框架在标准基准测试上以巨大优势超越了所有现有的联邦学习方法。