The key premise of federated learning (FL) is to train ML models across a diverse set of data-owners (clients), without exchanging local data. An overarching challenge to this date is client heterogeneity, which may arise not only from variations in data distribution, but also in data quality, as well as compute/communication latency. An integrated view of these diverse and concurrent sources of heterogeneity is critical; for instance, low-latency clients may have poor data quality, and vice versa. In this work, we propose FLASH(Federated Learning Across Simultaneous Heterogeneities), a lightweight and flexible client selection algorithm that outperforms state-of-the-art FL frameworks under extensive sources of heterogeneity, by trading-off the statistical information associated with the client's data quality, data distribution, and latency. FLASH is the first method, to our knowledge, for handling all these heterogeneities in a unified manner. To do so, FLASH models the learning dynamics through contextual multi-armed bandits (CMAB) and dynamically selects the most promising clients. Through extensive experiments, we demonstrate that FLASH achieves substantial and consistent improvements over state-of-the-art baselines -- as much as 10% in absolute accuracy -- thanks to its unified approach. Importantly, FLASH also outperforms federated aggregation methods that are designed to handle highly heterogeneous settings and even enjoys a performance boost when integrated with them.

翻译：摘要：联邦学习（FL）的核心前提是在不交换本地数据的情况下，跨多样化的数据持有方（客户端）训练机器学习模型。当前面临的首要挑战是客户端异质性，这种异质性不仅可能源于数据分布的差异，还可能来自数据质量以及计算/通信延迟的差异。对这些多样化且并发存在的异质性来源进行整合性审视至关重要：例如，低延迟客户端可能数据质量较差，反之亦然。在本工作中，我们提出FLASH（面向多重异质性的联邦学习），这是一种轻量级且灵活的客户端选择算法。通过权衡客户端数据质量、数据分布和延迟相关的统计信息，该算法在广泛的异质性来源下优于现有最先进的联邦学习框架。据我们所知，FLASH是首个能够统一处理所有此类异质性的方法。为此，FLASH通过上下文多臂老虎机（CMAB）对学习动态进行建模，并动态选择最具潜力的客户端。通过大量实验，我们证明得益于其统一方法，FLASH相较于最先进的基准方法实现了显著且一致的性能提升——绝对准确率提升高达10%。重要的是，FLASH在性能上甚至优于专为高度异质场景设计的联邦聚合方法，并且在与这些方法集成时还能进一步提升性能。