In federated learning, it is common to assume that clients are always available to participate in training, which may not be feasible with user devices in practice. Recent works analyze federated learning under more realistic participation patterns, such as cyclic client availability or arbitrary participation. However, all such works either require strong assumptions (e.g., all clients participate almost surely within a bounded window), do not achieve linear speedup and reduced communication rounds, or are not applicable in the general non-convex setting. In this work, we focus on nonconvex optimization and consider participation patterns in which the chance of participation over a fixed window of rounds is equal among all clients, which includes cyclic client availability as a special case. Under this setting, we propose a new algorithm, named Amplified SCAFFOLD, and prove that it achieves linear speedup, reduced communication, and resilience to data heterogeneity simultaneously. In particular, for cyclic participation, our algorithm is proved to enjoy $\mathcal{O}(\epsilon^{-2})$ communication rounds to find an $\epsilon$-stationary point in the non-convex stochastic setting. In contrast, the prior work under the same setting requires $\mathcal{O}(\kappa^2 \epsilon^{-4})$ communication rounds, where $\kappa$ denotes the data heterogeneity. Therefore, our algorithm significantly reduces communication rounds due to better dependency in terms of $\epsilon$ and $\kappa$. Our analysis relies on a fine-grained treatment of the nested dependence between client participation and errors in the control variates, which results in tighter guarantees than previous work. We also provide experimental results with (1) synthetic data and (2) real-world data with a large number of clients $(N = 250)$, demonstrating the effectiveness of our algorithm under periodic client participation.

翻译：在联邦学习中，通常假设客户端始终可参与训练，但这在实际用户设备场景中可能不可行。近期研究分析了更现实的参与模式下的联邦学习，例如周期性客户端可用性或任意参与模式。然而，所有此类研究要么需要强假设（例如所有客户端在有界窗口内几乎必然参与），要么无法实现线性加速与减少通信轮次，要么不适用于一般非凸场景。在本工作中，我们聚焦于非凸优化问题，考虑所有客户端在固定轮次窗口内参与概率相等的参与模式，其中周期性客户端可用性可作为特例。在此设定下，我们提出一种名为Amplified SCAFFOLD的新算法，并证明其能同时实现线性加速、减少通信及对数据异构性的鲁棒性。特别地，对于周期性参与模式，我们证明该算法在非凸随机设定中仅需$\mathcal{O}(\epsilon^{-2})$通信轮次即可找到$\epsilon$-稳定点。相比之下，相同设定下的现有研究需要$\mathcal{O}(\kappa^2 \epsilon^{-4})$通信轮次，其中$\kappa$表示数据异构性程度。因此，我们的算法凭借对$\epsilon$和$\kappa$更优的依赖关系显著减少了通信轮次。我们的分析依赖于对客户端参与与控制变量误差之间嵌套依赖关系的精细处理，从而获得了比先前研究更严格的收敛保证。我们还通过（1）合成数据与（2）大规模客户端（$N = 250$）的真实数据实验，验证了所提算法在周期性客户端参与模式下的有效性。