Federated learning (FL) is a distributed machine learning framework where the global model of a central server is trained via multiple collaborative steps by participating clients without sharing their data. While being a flexible framework, where the distribution of local data, participation rate, and computing power of each client can greatly vary, such flexibility gives rise to many new challenges, especially in the hyperparameter tuning on the client side. We propose $\Delta$-SGD, a simple step size rule for SGD that enables each client to use its own step size by adapting to the local smoothness of the function each client is optimizing. We provide theoretical and empirical results where the benefit of the client adaptivity is shown in various FL scenarios.

翻译：联邦学习（FL）是一种分布式机器学习框架，其中中央服务器的全局模型通过参与客户端的多轮协作训练而成，无需共享其数据。尽管该框架具有高度灵活性——各客户端的本地数据分布、参与率及计算能力可能存在显著差异——但这种灵活性也带来诸多新挑战，尤其在客户端超参数调优方面。我们提出$\Delta$-SGD，一种简单的SGD步长规则，使每个客户端能够通过自适应优化目标函数的局部平滑度来使用其自身的步长。我们提供了理论与实证结果，在各种联邦学习场景中验证了客户端自适应性的优势。