This paper focuses on reducing the communication cost of federated learning by exploring generalization bounds and representation learning. We first characterize a tighter generalization bound for one-round federated learning based on local clients' generalizations and heterogeneity of data distribution (non-iid scenario). We also characterize a generalization bound in R-round federated learning and its relation to the number of local updates (local stochastic gradient descents (SGDs)). Then, based on our generalization bound analysis and our representation learning interpretation of this analysis, we show for the first time that less frequent aggregations, hence more local updates, for the representation extractor (usually corresponds to initial layers) leads to the creation of more generalizable models, particularly for non-iid scenarios. We design a novel Federated Learning with Adaptive Local Steps (FedALS) algorithm based on our generalization bound and representation learning analysis. FedALS employs varying aggregation frequencies for different parts of the model, so reduces the communication cost. The paper is followed with experimental results showing the effectiveness of FedALS.

翻译：本文聚焦于通过探索泛化界和表示学习来降低联邦学习的通信成本。首先，我们基于局部客户端的泛化能力和数据分布异质性（非独立同分布场景）刻画了单轮联邦学习的更紧泛化界。同时，我们刻画了R轮联邦学习的泛化界及其与局部更新次数（局部随机梯度下降）的关系。随后，基于泛化界分析及其表示学习解释，我们首次证明：对特征提取器（通常对应于初始层）采用较低聚合频率（即更多局部更新）有助于生成更具泛化能力的模型，尤其在非独立同分布场景下。基于泛化界与表示学习分析，我们设计了一种新颖的自适应局部步长联邦学习（FedALS）算法。该算法针对模型不同部分采用差异化聚合频率，从而降低通信成本。最后通过实验验证了FedALS的有效性。