Both centralized and decentralized approaches have shown excellent performance and great application value in federated learning (FL). However, current studies do not provide sufficient evidence to show which one performs better. Although from the optimization perspective, decentralized methods can approach the comparable convergence of centralized methods with less communication, its test performance has always been inefficient in empirical studies. To comprehensively explore their behaviors in FL, we study their excess risks, including the joint analysis of both optimization and generalization. We prove that on smooth non-convex objectives, 1) centralized FL (CFL) always generalizes better than decentralized FL (DFL); 2) from perspectives of the excess risk and test error in CFL, adopting partial participation is superior to full participation; and, 3) there is a necessary requirement for the topology in DFL to avoid performance collapse as the training scale increases. Based on some simple hardware metrics, we could evaluate which framework is better in practice. Extensive experiments are conducted on common setups in FL to validate that our theoretical analysis is contextually valid in practical scenarios.

翻译：集中式和分布式方法都在联邦学习（FL）中展现了优异的性能和巨大的应用价值。然而，现有研究并未提供充分证据表明哪一种表现更优。尽管从优化角度来看，分布式方法能以更少的通信量达到与集中式方法相当的可比收敛效果，但其测试性能在实证研究中始终不够高效。为全面探究它们在FL中的行为，我们研究了其超额风险，包括对优化和泛化的联合分析。我们证明，在光滑非凸目标上：1）集中式联邦学习（CFL）的泛化性能始终优于分布式联邦学习（DFL）；2）从超额风险和CFL测试误差的角度来看，采用部分参与优于全参与；3）DFL中拓扑结构必须满足必要要求，以避免随着训练规模扩大出现性能崩溃。基于一些简单的硬件指标，我们可以评估实践中哪种框架更优。我们在FL的常见设置上进行了大量实验，验证了我们的理论分析在实际场景中具有情境有效性。