Federated learning (FL) enables collaborative model training among distributed devices without data sharing, but existing FL suffers from poor scalability because of global model synchronization. To address this issue, hierarchical federated learning (HFL) has been recently proposed to let edge servers aggregate models of devices in proximity, while synchronizing via the cloud periodically. However, a critical open challenge about how to make a good synchronization scheme (when devices and edges should be synchronized) is still unsolved. Devices are heterogeneous in computing and communication capability, and their data could be non-IID. No existing work can well synchronize various roles (\textit{e.g.}, devices and edges) in HFL to guarantee high learning efficiency and accuracy. In this paper, we propose a learning-based synchronization scheme for HFL systems. By collecting data such as edge models, CPU usage, communication time, \textit{etc}., we design a deep reinforcement learning-based approach to decide the frequencies of cloud aggregation and edge aggregation, respectively. The proposed scheme well considers device heterogeneity, non-IID data and device mobility, to maximize the training model accuracy while minimizing the energy overhead. Meanwhile, the convergence bound of the proposed synchronization scheme has been analyzed. And we build an HFL testbed and conduct the experiments with real data obtained from Raspberry Pi and Alibaba Cloud. Extensive experiments under various settings are conducted to confirm the effectiveness of \textit{Arena}.

翻译：摘要：联邦学习（Federated Learning, FL）通过分布式设备在不共享数据的情况下实现协同模型训练，但由于全局模型同步的限制，现有FL的可扩展性较差。为解决该问题，近年来提出了层次化联邦学习（Hierarchical Federated Learning, HFL），该方法让边缘服务器聚合邻近设备的模型，并通过云端定期进行同步。然而，如何制定高效的同步方案（即确定设备与边缘服务器何时同步）仍是一个关键未解挑战。设备在计算与通信能力上具有异构性，且其数据可能存在非独立同分布（non-IID）特性。现有工作均无法在HFL中有效协调不同角色（如设备与边缘服务器）的同步，从而难以保证高学习效率与准确性。本文提出一种基于学习的HFL系统同步方案。通过收集边缘模型、CPU使用率、通信时间等数据，我们设计了一种基于深度强化学习的方法，分别决定云聚合与边缘聚合的频率。该方案充分考虑了设备异构性、非IID数据以及设备移动性，以最大化训练模型精度并最小化能量开销。同时，我们分析了所提同步方案的收敛界，并基于树莓派（Raspberry Pi）与阿里云（Alibaba Cloud）的实际数据搭建了HFL测试平台进行实验。在不同设置下的广泛实验验证了Arena方案的有效性。