Federated Machine Learning (FL) has received considerable attention in recent years. FL benchmarks are predominantly explored in either simulated systems or data center environments, neglecting the setups of real-world systems, which are often closely linked to edge computing. We close this research gap by introducing FLEdge, a benchmark targeting FL workloads in edge computing systems. We systematically study hardware heterogeneity, energy efficiency during training, and the effect of various differential privacy levels on training in FL systems. To make this benchmark applicable to real-world scenarios, we evaluate the impact of client dropouts on state-of-the-art FL strategies with failure rates as high as 50%. FLEdge provides new insights, such as that training state-of-the-art FL workloads on older GPU-accelerated embedded devices is up to 3x more energy efficient than on modern server-grade GPUs.

翻译：联邦机器学习近年来受到广泛关注。现有联邦学习基准测试主要在模拟系统或数据中心环境中开展，忽视了通常与边缘计算紧密关联的真实系统部署场景。为填补这一研究空白，我们提出FLEdge——一个面向边缘计算系统中联邦学习工作负载的基准测试框架。我们系统性地研究了联邦学习系统中的硬件异构性、训练过程中的能源效率，以及不同差分隐私级别对训练效果的影响。为使该基准测试适用于真实场景，我们评估了高达50%的客户端退出率对当前最先进联邦学习策略的影响。FLEdge提供了新的见解，例如在老旧GPU加速嵌入式设备上训练最先进的联邦学习工作负载，其能效比在现代化服务器级GPU上最高可提升3倍。