We consider federated learning in tiered communication networks. Our network model consists of a set of silos, each holding a vertical partition of the data. Each silo contains a hub and a set of clients, with the silo's vertical data shard partitioned horizontally across its clients. We propose Tiered Decentralized Coordinate Descent (TDCD), a communication-efficient decentralized training algorithm for such two-tiered networks. The clients in each silo perform multiple local gradient steps before sharing updates with their hub to reduce communication overhead. Each hub adjusts its coordinates by averaging its workers' updates, and then hubs exchange intermediate updates with one another. We present a theoretical analysis of our algorithm and show the dependence of the convergence rate on the number of vertical partitions and the number of local updates. We further validate our approach empirically via simulation-based experiments using a variety of datasets and objectives.

翻译：我们研究分层通信网络中的联邦学习。所提出的网络模型由一组孤岛组成，每个孤岛持有数据的垂直分区，且每个孤岛包含一个中心节点和一组客户端，其中该孤岛的垂直数据分片在客户端之间水平分布。我们提出了一种面向此类双层网络的通信高效去中心化训练算法——分层去中心化坐标下降法（TDCD）。每个孤岛内的客户端在执行多次局部梯度步骤后，才向中心节点共享更新，从而降低通信开销。各中心节点通过平均其工作节点的更新来调整自身坐标，随后各中心节点之间交换中间更新。我们从理论上分析了该算法，并展示了收敛速率与垂直分区数量及局部更新次数之间的依赖关系。我们进一步通过基于模拟的实验（使用多种数据集和目标函数）从经验上验证了该方法。