Federated Learning (FL) aims to train high-quality models in collaboration with distributed clients while not uploading their local data, which attracts increasing attention in both academia and industry. However, there is still a considerable gap between the flourishing FL research and real-world scenarios, mainly caused by the characteristics of heterogeneous devices and its scales. Most existing works conduct evaluations with homogeneous devices, which are mismatched with the diversity and variability of heterogeneous devices in real-world scenarios. Moreover, it is challenging to conduct research and development at scale with heterogeneous devices due to limited resources and complex software stacks. These two key factors are important yet underexplored in FL research as they directly impact the FL training dynamics and final performance, making the effectiveness and usability of FL algorithms unclear. To bridge the gap, in this paper, we propose an efficient and scalable prototyping system for real-world cross-device FL, FS-Real. It supports heterogeneous device runtime, contains parallelism and robustness enhanced FL server, and provides implementations and extensibility for advanced FL utility features such as personalization, communication compression and asynchronous aggregation. To demonstrate the usability and efficiency of FS-Real, we conduct extensive experiments with various device distributions, quantify and analyze the effect of the heterogeneous device and various scales, and further provide insights and open discussions about real-world FL scenarios. Our system is released to help to pave the way for further real-world FL research and broad applications involving diverse devices and scales.

翻译：联邦学习（FL）旨在通过与分布式客户端协作训练高质量模型，同时不上传其本地数据，这引起了学术界和工业界越来越多的关注。然而，蓬勃发展的FL研究与现实场景之间仍存在显著差距，这主要是由异构设备的特性及其规模所致。现有大多数研究采用同构设备进行评估，这与现实场景中异构设备的多样性和可变性不相匹配。此外，由于资源有限且软件栈复杂，在异构设备上大规模开展研发工作极具挑战性。这两个关键因素在FL研究中至关重要却未得到充分探索，因为它们直接影响FL的训练动态和最终性能，使得FL算法的有效性和可用性变得不明确。为弥合这一差距，本文提出了一种面向真实世界跨设备FL的高效可扩展原型系统FS-Real。该系统支持异构设备运行时环境，包含并行性和鲁棒性增强的FL服务器，并提供个性化、通信压缩和异步聚合等高级FL实用功能的实现与可扩展性。为展示FS-Real的可用性和效率，我们开展了涵盖多种设备分布的广泛实验，量化并分析了异构设备及不同规模的影响，进而针对真实世界的FL场景提供了深入见解和开放式讨论。我们的系统已开源发布，旨在为涉及多样化设备与规模的真实世界FL研究及广泛应用铺平道路。