There are two categories of methods in Federated Learning (FL) for joint training across multiple clients: i) parallel FL (PFL), where clients train models in a parallel manner; and ii) sequential FL (SFL), where clients train models in a sequential manner. In contrast to that of PFL, the convergence theory of SFL on heterogeneous data is still lacking. In this paper, we establish the convergence guarantees of SFL for strongly/general/non-convex objectives on heterogeneous data. The convergence guarantees of SFL are better than that of PFL on heterogeneous data with both full and partial client participation. Experimental results validate the counterintuitive analysis result that SFL outperforms PFL on extremely heterogeneous data in cross-device settings.

翻译：联邦学习（FL）中跨多个客户端联合训练的方法可分为两类：i）并行联邦学习（PFL），客户端以并行方式训练模型；ii）序列联邦学习（SFL），客户端以序列方式训练模型。与PFL不同，异构数据上SFL的收敛理论仍尚不完善。本文针对异构数据上的强凸/一般凸/非凸目标函数，建立了SFL的收敛性保证。在客户端全部参与和部分参与两种场景下，SFL在异构数据上的收敛性保证均优于PFL。实验结果验证了这一反直觉的分析结论：在跨设备场景下的极端异构数据中，SFL性能优于PFL。