This work considers the category distribution heterogeneity in federated learning. This issue is due to biased labeling preferences at multiple clients and is a typical setting of data heterogeneity. To alleviate this issue, most previous works consider either regularizing local models or fine-tuning the global model, while they ignore the adjustment of aggregation weights and simply assign weights based on the dataset size. However, based on our empirical observations and theoretical analysis, we find that the dataset size is not optimal and the discrepancy between local and global category distributions could be a beneficial and complementary indicator for determining aggregation weights. We thus propose a novel aggregation method, Federated Learning with Discrepancy-aware Collaboration (FedDisco), whose aggregation weights not only involve both the dataset size and the discrepancy value, but also contribute to a tighter theoretical upper bound of the optimization error. FedDisco also promotes privacy-preservation, communication and computation efficiency, as well as modularity. Extensive experiments show that our FedDisco outperforms several state-of-the-art methods and can be easily incorporated with many existing methods to further enhance the performance. Our code will be available at https://github.com/MediaBrain-SJTU/FedDisco.

翻译：本文考虑联邦学习中的类别分布异质性。该问题源于多个客户端存在有偏的标签偏好，是数据异质性的一种典型场景。为缓解此问题，以往多数工作要么对局部模型进行正则化，要么对全局模型进行微调，但忽略了聚合权重的调整，仅基于数据集大小分配权重。然而，基于经验观察与理论分析，我们发现数据集大小并非最优选择，局部与全局类别分布之间的差异可作为确定聚合权重的一个有益且互补的指标。为此，我们提出一种新的聚合方法——基于差异感知协作的联邦学习（FedDisco），其聚合权重不仅同时包含数据集大小与差异值，还有助于收紧优化误差的理论上界。FedDisco还促进了隐私保护、通信与计算效率以及模块化。大量实验表明，我们的FedDisco优于多种现有最优方法，并易于与现有方法集成以进一步提升性能。我们的代码将开源至https://github.com/MediaBrain-SJTU/FedDisco。