This paper investigates federated learning (FL) in a multi-hop communication setup, such as in constellations with inter-satellite links. In this setup, part of the FL clients are responsible for forwarding other client's results to the parameter server. Instead of using conventional routing, the communication efficiency can be improved significantly by using in-network model aggregation at each intermediate hop, known as incremental aggregation (IA). Prior works [1] have indicated diminishing gains for IA under gradient sparsification. Here we study this issue and propose several novel correlated sparsification methods for IA. Numerical results show that, for some of these algorithms, the full potential of IA is still available under sparsification without impairing convergence. We demonstrate a 15x improvement in communication efficiency over conventional routing and a 11x improvement over state-of-the-art (SoA) sparse IA.

翻译：本文研究了多跳通信场景下的联邦学习（FL），例如具有星间链路的卫星星座。在此场景中，部分FL客户端负责将其他客户端的结果转发给参数服务器。相较于使用传统路由，通过在每一跳中间节点进行网络内模型聚合（称为增量聚合（IA）），可以显著提高通信效率。先前的研究[1]表明，在梯度稀疏化条件下，IA的增益会减弱。本文针对此问题进行研究，并提出了几种适用于IA的新型相关稀疏化方法。数值结果表明，对于其中一些算法，在稀疏化条件下，IA的全部潜力仍然得以保留，且不影响收敛性。我们证明了所提方法相较于传统路由实现了15倍的通信效率提升，相较于最先进的稀疏IA方法实现了11倍的提升。