There are two paradigms in Federated Learning (FL): parallel FL (PFL), where models are trained in a parallel manner across clients; and sequential FL (SFL), where models are trained in a sequential manner across clients. In contrast to that of PFL, the convergence theory of SFL on heterogeneous data is still lacking. To resolve the theoretical dilemma of SFL, we establish sharp convergence guarantees for SFL on heterogeneous data with both upper and lower bounds. Specifically, we derive the upper bounds for strongly convex, general convex and non-convex objective functions, and construct the matching lower bounds for the strongly convex and general convex objective functions. Then, we compare the upper bounds of SFL with those of PFL, showing that SFL outperforms PFL (at least, when the level of heterogeneity is relatively high). Experimental results on quadratic functions and real data sets validate the counterintuitive comparison result.

翻译：联邦学习存在两种范式：并行联邦学习（PFL）在各客户端以并行方式训练模型；顺序联邦学习（SFL）在各客户端以顺序方式训练模型。相较于并行联邦学习，顺序联邦学习在异构数据上的收敛理论仍然缺乏。为解决该理论困境，我们为异构数据上的顺序联邦学习建立了包含上界和下界的严格收敛保证。具体而言，我们推导了强凸、一般凸和非凸目标函数的上界，并构建了强凸和一般凸目标函数的匹配下界。随后，通过比较顺序联邦学习与并行联邦学习的上界，表明顺序联邦学习的性能优于并行联邦学习（至少在异构程度较高的情况下如此）。二次函数和真实数据集的实验结果验证了这一反直觉的比较结论。