Federated learning (FL) is a distributed machine learning framework where the global model of a central server is trained via multiple collaborative steps by participating clients without sharing their data. While being a flexible framework, where the distribution of local data, participation rate, and computing power of each client can greatly vary, such flexibility gives rise to many new challenges, especially in the hyperparameter tuning on both the server and the client side. We propose $\Delta$-SGD, a simple step size rule for SGD that enables each client to use its own step size by adapting to the local smoothness of the function each client is optimizing. We provide theoretical and empirical results where the benefit of the client adaptivity is shown in various FL scenarios. In particular, our proposed method achieves TOP-1 accuracy in 73% and TOP-2 accuracy in 100% of the experiments considered without additional tuning.

翻译：联邦学习（FL）是一种分布式机器学习框架，其中中央服务器的全局模型通过参与客户端的多次协作步骤进行训练，而无需共享其数据。尽管FL是一个灵活的框架，其中本地数据分布、参与率和每个客户端的计算能力可能存在显著差异，但这种灵活性也带来了许多新挑战，尤其是在服务器端和客户端的超参数调优方面。我们提出了$\Delta$-SGD，一种简单的SGD步长规则，使每个客户端能够通过适应其优化函数的局部平滑度来使用自己的步长。我们提供了理论和实证结果，展示了客户端自适应性在各种联邦学习场景中的优势。特别地，在无需额外调优的情况下，所提方法在73%的实验案例中达到TOP-1准确率，并在100%的实验案例中达到TOP-2准确率。