In the setting of federated optimization, where a global model is aggregated periodically, step asynchronism occurs when participants conduct model training by efficiently utilizing their computational resources. It is well acknowledged that step asynchronism leads to objective inconsistency under non-i.i.d. data, which degrades the model's accuracy. To address this issue, we propose a new algorithm FedaGrac, which calibrates the local direction to a predictive global orientation. Taking advantage of the estimated orientation, we guarantee that the aggregated model does not excessively deviate from the global optimum while fully utilizing the local updates of faster nodes. We theoretically prove that FedaGrac holds an improved order of convergence rate than the state-of-the-art approaches and eliminates the negative effect of step asynchronism. Empirical results show that our algorithm accelerates the training and enhances the final accuracy.

翻译：在联邦优化场景中，全局模型被周期性地聚合，当参与者通过高效利用其计算资源进行模型训练时，步长异步性随之产生。众所周知，在非独立同分布数据下，步长异步性会导致目标不一致性，从而降低模型精度。为解决该问题，我们提出新算法FedaGrac，该算法将局部方向校准至预测的全局方向。借助这一估计方向，我们能够在充分利用快速节点局部更新的同时，保证聚合模型不会过度偏离全局最优值。从理论上证明，FedaGrac的收敛速度阶数优于现有最优方法，并消除了步长异步性的负面影响。实验结果表明，我们的算法加速了训练过程并提升了最终精度。